Oct 30 00:38:41.905185 ip-10-0-141-17 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 00:38:41.905196 ip-10-0-141-17 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 00:38:41.905203 ip-10-0-141-17 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 00:38:41.905462 ip-10-0-141-17 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 00:38:52.061594 ip-10-0-141-17 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 00:38:52.061614 ip-10-0-141-17 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ed687dff44da462096f7dd80fd38efd6 --
Oct 30 00:40:51.049174 ip-10-0-141-17 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 00:40:51.471105 ip-10-0-141-17 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:40:51.471105 ip-10-0-141-17 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 00:40:51.471105 ip-10-0-141-17 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:40:51.471105 ip-10-0-141-17 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:40:51.476367 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476244    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.476367 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476366    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476370    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476374    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476378    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476381    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476384    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476387    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476391    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476395    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476398    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476401    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476403    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476406    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476409    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476412    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476414    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476417    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476427    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476430    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476433    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.476437 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476436    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476438    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476441    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476444    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476446    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476450    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476452    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476455    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476457    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476460    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476463    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476466    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476468    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476471    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476473    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476476    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476478    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476481    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476483    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476486    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.476929 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476489    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476491    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476494    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476496    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476499    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476501    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476504    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476507    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476510    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476512    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476515    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476517    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476520    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476523    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476526    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476529    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476531    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476534    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476537    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476539    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.477399 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476542    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476545    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476547    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476551    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476555    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476558    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476561    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476564    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476566    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476569    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476571    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476574    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476577    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476579    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476581    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476584    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476586    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476589    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476591    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.477880 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476594    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476597    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476603    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476606    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476608    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476611    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476614    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476616    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476619    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476622    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476624    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476627    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476629    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476632    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476634    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476637    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476639    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476641    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476644    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.478361 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476646    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476649    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476651    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476655    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476657    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476660    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.476662    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477500    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477508    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477512    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477515    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477518    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477520    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477524    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477527    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477530    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477533    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477536    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477539    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.479503 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477541    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477545    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477547    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477550    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477553    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477555    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477558    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477560    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477563    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477566    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477568    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477571    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477573    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477576    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477578    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477581    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477584    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477586    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477589    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477591    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.479972 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477593    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477596    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477599    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477601    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477604    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477608    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477612    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477616    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477619    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477621    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477625    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477627    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477630    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477633    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477635    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477638    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477641    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477644    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477646    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.480453 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477649    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477651    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477653    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477656    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477658    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477661    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477663    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477666    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477668    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477671    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477673    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477676    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477678    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477681    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477684    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477687    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477689    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477692    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477694    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477697    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.480924 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477700    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477703    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477705    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477708    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477711    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477713    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477716    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477718    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477721    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477724    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477726    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477728    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477731    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477734    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477736    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477739    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477741    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477743    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477746    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477748    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.481401 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477750    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477753    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477755    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477758    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477760    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477762    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477765    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477768    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477772    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477776    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477779    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477782    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477784    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477787    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.477789    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478273    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478279    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478282    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478284    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.481948 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478287    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478290    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478293    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478295    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478298    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478300    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478303    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478306    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478308    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478311    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478313    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478316    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478318    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478321    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478323    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478326    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478329    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478331    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478334    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478338    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.482501 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478342    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478344    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478347    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478350    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478352    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478355    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478357    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478360    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478362    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478365    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478368    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478371    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478373    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478376    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478378    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478381    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478383    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478386    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478388    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478390    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.483011 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478393    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478395    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478397    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478400    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478402    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478405    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478407    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478410    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478412    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478415    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478418    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478422    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478425    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478428    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478430    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478433    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478436    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478438    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478440    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.483483 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478443    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478445    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478448    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478450    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478454    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478456    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478459    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478461    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478464    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478466    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478468    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478471    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478473    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478476    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478478    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478481    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478483    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478486    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478488    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.483999 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478491    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478493    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478496    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478499    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478501    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478506    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478509    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478512    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478515    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478517    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478520    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478523    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478525    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478527    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478530    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478532    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478535    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478537    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478541    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478543    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.484456 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478546    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478548    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478553    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.478557    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479443    2580 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479455    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479463    2580 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479468    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479473    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479476    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479480    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479485    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479488    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479491    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479495    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479498    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479502    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479507    2580 flags.go:64] FLAG: --cgroup-root=""
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479510    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479513    2580 flags.go:64] FLAG: --client-ca-file=""
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479515    2580 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479519    2580 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479523    2580 flags.go:64] FLAG: --cluster-domain=""
Oct 30 00:40:51.484951 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479526    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479529    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479532    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479535    2580 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479539    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479542    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479545    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479549    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479553    2580 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479556    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479559    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479561    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479565    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479569    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479572    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479575    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479580    2580 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479583    2580 flags.go:64] FLAG: --enable-server="true"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479585    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479590    2580 flags.go:64] FLAG: --event-burst="100"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479593    2580 flags.go:64] FLAG: --event-qps="50"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479596    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479599    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479602    2580 flags.go:64] FLAG: --eviction-hard=""
Oct 30 00:40:51.485509 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479607    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479609    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479614    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479617    2580 flags.go:64] FLAG: --eviction-soft=""
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479620    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479623    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479626    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479629    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479632    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479634    2580 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479637    2580 flags.go:64] FLAG: --feature-gates=""
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479641    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479644    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479647    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479650    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479653    2580 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479656    2580 flags.go:64] FLAG: --help="false"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479660    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479663    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479666    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479669    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479672    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479675    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479678    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 00:40:51.486110 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479682    2580 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479685    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479689    2580 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479692    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479695    2580 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479697    2580 flags.go:64] FLAG: --kube-reserved=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479700    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479703    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479706    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479709    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479713    2580 flags.go:64] FLAG: --lock-file=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479716    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479719    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479722    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479728    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479730    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479733    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479736    2580 flags.go:64] FLAG: --logging-format="text"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479739    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479742    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479745    2580 flags.go:64] FLAG: --manifest-url=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479748    2580 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479756    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479759    2580 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479763    2580 flags.go:64] FLAG: --max-pods="110"
Oct 30 00:40:51.486692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479766    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479786    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479790    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479793    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479796    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479799    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479802    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479811    2580 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479814    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479817    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479820    2580 flags.go:64] FLAG: --pod-cidr=""
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479823    2580 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479826    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479829    2580 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479832    2580 flags.go:64] FLAG: --port="10250"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479835    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479838    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e9f9c43c57ad8f2"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479856    2580 flags.go:64] FLAG: --qos-reserved=""
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479860    2580 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479863    2580 flags.go:64] FLAG: --register-node="true"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479866    2580 flags.go:64] FLAG: --register-with-taints=""
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479869    2580 flags.go:64] FLAG: --registry-burst="10"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479872    2580 flags.go:64] FLAG: --registry-qps="5"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479875    2580 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479878    2580 flags.go:64] FLAG: --reserved-memory=""
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479881    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 00:40:51.487412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479884    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479887    2580 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479890    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479893    2580 flags.go:64] FLAG: --runonce="false"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479896    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479899    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479902    2580 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479905    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479908    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479911    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479914    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479918    2580 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479921    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479924    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479927    2580 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479930    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479933    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479936    2580 flags.go:64] FLAG: --system-cgroups=""
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479939    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479945    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479948    2580 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479951    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479955    2580 flags.go:64] FLAG: --tls-min-version=""
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479959    2580 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479962    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 00:40:51.488407 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479965    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479969    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479972    2580 flags.go:64] FLAG: --v="2"
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479976    2580 flags.go:64] FLAG: --version="false"
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479981    2580 flags.go:64] FLAG: --vmodule=""
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479985    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.479988    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480130    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480135    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480138    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480142    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480144    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480147    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480149    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480152    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480154    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480157    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480160    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480162    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480165    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480169    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.489071 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480171    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480174    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480176    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480180    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480184    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480187    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480190    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480192    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480194    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480199    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480202    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480204    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480207    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480210    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480212    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480215    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480217    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480220    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480223    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.489567 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480225    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480228    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480230    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480233    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480235    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480238    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480240    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480242    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480245    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480248    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480250    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480253    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480257    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480259    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480263    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480267    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480270    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480273    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480276    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.490054 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480278    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480281    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480283    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480287    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480289    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480292    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480295    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480297    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480300    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480302    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480305    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480307    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480309    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480312    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480314    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480317    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480319    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480322    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480324    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480326    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.490498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480329    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480332    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480334    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480337    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480339    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480343    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480354    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480357    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480359    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480361    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480364    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480366    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480369    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480371    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480374    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480377    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480379    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480382    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480385    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480387    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.490970 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480390    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480392    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480394    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480397    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480400    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480402    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480405    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480407    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480410    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480412    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480415    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480417    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480420    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480422    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.480450    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480486    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.491439 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480492    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480496    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480499    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480502    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480504    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480506    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480509    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480512    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480514    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480517    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480519    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480522    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480526    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480529    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480532    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480534    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480538    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480541    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480545    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.491809 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480548    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480550    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480553    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480555    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480557    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480560    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480563    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480566    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480568    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480571    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480573    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480576    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480578    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480581    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480586    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480588    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480591    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480593    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480596    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.492267 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480598    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480601    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480603    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480606    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480609    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480612    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480616    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480619    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480622    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480625    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480629    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480632    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480634    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480637    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480639    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480641    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480644    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480646    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480649    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.492716 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480651    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480654    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480656    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480659    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480661    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480664    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480666    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480669    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480674    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480677    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480679    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480682    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480684    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480687    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480689    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480692    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480694    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480697    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480699    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480703    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480706    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.493252 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480708    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480711    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480713    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480716    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480718    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480721    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480723    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480726    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480728    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480731    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480733    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480736    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480738    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480740    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480743    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480745    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480748    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480750    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480752    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480757    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.493743 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480759    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480762    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480765    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480768    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480770    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480772    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.480775    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.488969    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.488986    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.489498    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.489504    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489562    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489568    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489571    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489574    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489577    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489580    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489582    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489585    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489588    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.494278 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489590    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489593    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489596    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489599    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489601    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489604    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489606    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489609    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489612    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489614    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489617    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489619    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489621    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489624    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489627    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489629    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489632    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489635    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489638    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.494753 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489640    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489645    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489650    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489653    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489657    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489660    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489662    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489665    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489667    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489670    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489672    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489674    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489677    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489679    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489682    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489684    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489687    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489690    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489692    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.495207 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489695    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489698    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489700    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489703    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489705    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489708    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489711    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489713    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489716    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489718    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489720    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489723    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489725    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489728    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489731    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489733    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489736    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489738    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489741    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.495684 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489743    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489745    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489748    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489750    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489753    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489755    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489758    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489760    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489762    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489765    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489767    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489770    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489772    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489775    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489778    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489781    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489783    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489787    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489791    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.496143 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489794    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489796    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489799    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489801    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489804    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489806    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489808    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489811    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489813    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489816    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489818    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489821    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489823    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489826    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489828    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489830    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489833    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489836    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489838    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489854    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.496595 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.489856    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.489886    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490031    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490036    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490039    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490041    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490044    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490047    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490050    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490052    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490055    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490057    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490060    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490063    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490066    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.497196 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490068    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490070    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490073    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490075    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490077    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490080    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490083    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490085    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490088    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490090    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490092    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490095    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490097    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490100    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490102    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490104    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490107    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490110    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490112    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490114    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.497546 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490117    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490119    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490121    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490124    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490126    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490129    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490131    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490134    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490137    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490140    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490142    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490144    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490147    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490149    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490152    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490154    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490156    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490159    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490161    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490164    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.498022 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490166    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490169    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490171    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490174    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490176    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490178    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490181    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490183    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490185    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490188    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490190    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490192    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490195    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490197    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490199    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490202    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490205    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490207    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490213    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.498498 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490216    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490219    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490221    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490224    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490227    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490229    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490232    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490234    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490236    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490239    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490242    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490244    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490247    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490250    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490253    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490256    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490258    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490262    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490266    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.498953 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490269    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490272    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490275    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490277    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490279    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490282    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490284    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490286    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490289    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490291    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490294    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490296    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490301    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490307    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490310    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.490334    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:40:51.499387 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490365    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490369    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490372    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490375    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490377    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490380    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490383    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490385    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490388    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490390    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490393    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490396    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490398    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490401    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490403    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490406    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490408    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490411    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490414    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:40:51.499785 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490416    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490419    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490421    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490424    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490426    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490429    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490433    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490435    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490438    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490440    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490445    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490450    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490453    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490456    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490459    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490461    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490464    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490466    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490468    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:40:51.500269 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490471    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490473    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490476    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490478    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490480    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490483    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490485    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490488    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490490    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490493    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490496    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490498    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490501    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490503    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490506    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490509    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490511    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490514    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490518    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:40:51.500733 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490520    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490522    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490525    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490527    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490531    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490533    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490536    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490538    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490541    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490543    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490546    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490548    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490551    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490553    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490555    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490558    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490560    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490563    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490565    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490567    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:40:51.501326 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490570    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490572    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490575    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490577    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490580    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490582    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490585    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490587    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490589    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490592    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490595    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490598    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490601    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490603    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490606    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490608    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490611    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490614    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490616    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490619    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:40:51.501858 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490621    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490624    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490626    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490629    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490631    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490635    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490637    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490639    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:51.490642    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.490704    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.492759    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.493714    2580 server.go:1014] "Starting client certificate rotation"
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.493807    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.493857    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.495566    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:40:51.502330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.498205    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:40:51.508112 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.508095    2580 log.go:25] "Validated CRI v1 runtime API"
Oct 30 00:40:51.511400 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.511386    2580 log.go:25] "Validated CRI v1 image API"
Oct 30 00:40:51.512224 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.512200    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 00:40:51.516897 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.516870    2580 fs.go:135] Filesystem UUIDs: map[3dc1538c-df45-4d3b-97b9-8b2a072572b2:/dev/nvme0n1p3 62064635-10b0-41da-984b-0e40437c29ae:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 00:40:51.516982 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.516895    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 00:40:51.523349 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.523231    2580 manager.go:217] Machine: {Timestamp:2026-10-30 00:40:51.521278713 +0000 UTC m=+0.353717531 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2985309 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2224c25fe8109cdd7355f2bd2d3b9a SystemUUID:ec2224c2-5fe8-109c-dd73-55f2bd2d3b9a BootID:ed687dff-44da-4620-96f7-dd80fd38efd6 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d3:9a:bc:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d3:9a:bc:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:8a:5b:85:78:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 00:40:51.523423 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.523351    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 00:40:51.523486 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.523472    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 00:40:51.525483 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.525454    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 00:40:51.525630 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.525484    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 00:40:51.525672 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.525637    2580 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 00:40:51.525672 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.525645    2580 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 00:40:51.525720 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.525686    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:40:51.525720 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.525709    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:40:51.526498 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.526487    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 00:40:51.530147 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.530094    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 00:40:51.530290 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.530280    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 00:40:51.531149 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.531138    2580 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 00:40:51.531186 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.531159    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 00:40:51.531186 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.531169    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 00:40:51.531186 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.531179    2580 kubelet.go:405] "Adding apiserver pod source"
Oct 30 00:40:51.531310 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.531187    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 00:40:51.532216 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.532192    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:40:51.534792 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.534774    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 00:40:51.539456 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.539443    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:40:51.540271 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.540258    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 00:40:51.540305 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.540289    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 00:40:51.541354 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.541343    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 00:40:51.541393 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.541364    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 00:40:51.541393 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.541374    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 00:40:51.541393 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.541380    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 00:40:51.541393 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.541386    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 00:40:51.541393 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.541392    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 00:40:51.542040 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542031    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 00:40:51.542077 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542041    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 00:40:51.542077 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542048    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 00:40:51.542077 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542055    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 00:40:51.542077 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542063    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 00:40:51.542077 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542072    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 00:40:51.542703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542694    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 00:40:51.542736 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.542707    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 00:40:51.546731 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.546710    2580 server.go:1263] "Started kubelet"
Oct 30 00:40:51.546796 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.546774    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 00:40:51.546872 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.546807    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 00:40:51.546907 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.546897    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 00:40:51.547510 ip-10-0-141-17 systemd[1]: Started Kubernetes Kubelet.
Oct 30 00:40:51.549688 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.549669    2580 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 00:40:51.551176 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.551160    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 00:40:51.553960 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.553943    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 00:40:51.556318 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.556304    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 00:40:51.556397 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.556316    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 00:40:51.556448 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.556400    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:51.556990 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.556975    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 00:40:51.557086 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.556998    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 00:40:51.557086 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557085    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 00:40:51.557243 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557209    2580 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 00:40:51.557243 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557218    2580 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 00:40:51.557243 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557239    2580 factory.go:55] Registering systemd factory
Oct 30 00:40:51.557386 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557255    2580 factory.go:223] Registration of the systemd container factory successfully
Oct 30 00:40:51.557473 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557458    2580 factory.go:153] Registering CRI-O factory
Oct 30 00:40:51.557534 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557475    2580 factory.go:223] Registration of the crio container factory successfully
Oct 30 00:40:51.557534 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557524    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 00:40:51.557659 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557540    2580 factory.go:103] Registering Raw factory
Oct 30 00:40:51.557659 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.557553    2580 manager.go:1199] Started watching for new ooms in manager
Oct 30 00:40:51.558505 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.558490    2580 manager.go:319] Starting recovery of all containers
Oct 30 00:40:51.571133 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.570351    2580 manager.go:324] Recovery completed
Oct 30 00:40:51.571738 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.571720    2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 30 00:40:51.574571 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.574553    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.576342 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576325    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.576422 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576371    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.576422 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576386    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.576908 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576892    2580 cpu_manager.go:226] "Starting" policy="none"
Oct 30 00:40:51.576978 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576908    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 00:40:51.576978 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576931    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 00:40:51.576978 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.576911    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.577704 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.577683    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.577779 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.577711    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.577779 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.577720    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.579102 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.578061    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-17.ec2.internal.18e327d024bda327  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-17.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-17.ec2.internal,},FirstTimestamp:2026-10-30 00:40:51.546678055 +0000 UTC m=+0.379116873,LastTimestamp:2026-10-30 00:40:51.546678055 +0000 UTC m=+0.379116873,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-17.ec2.internal,}"
Oct 30 00:40:51.579182 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.579123    2580 policy_none.go:50] "Start"
Oct 30 00:40:51.579182 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.579136    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 00:40:51.579182 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.579145    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 00:40:51.581054 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.581043    2580 policy_none.go:44] "Start"
Oct 30 00:40:51.588972 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.588833    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 00:40:51.589040 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.588880    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 00:40:51.595966 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.595950    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 00:40:51.596024 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.595989    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rkb84"
Oct 30 00:40:51.596933 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.596911    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 00:40:51.596999 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.596975    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 00:40:51.597042 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.596975    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 00:40:51.597082 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.596989    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-17.ec2.internal.18e327d026826efa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-17.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-17.ec2.internal,},FirstTimestamp:2026-10-30 00:40:51.576352506 +0000 UTC m=+0.408791312,LastTimestamp:2026-10-30 00:40:51.576352506 +0000 UTC m=+0.408791312,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-17.ec2.internal,}"
Oct 30 00:40:51.608164 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.608142    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rkb84"
Oct 30 00:40:51.614336 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.614305    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 00:40:51.616858 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.616828    2580 manager.go:341] "Starting Device Plugin manager"
Oct 30 00:40:51.616946 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.616874    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 00:40:51.616946 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.616886    2580 server.go:87] "Starting device plugin registration server"
Oct 30 00:40:51.617108 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.617094    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 00:40:51.617151 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.617119    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 00:40:51.617204 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.617190    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 00:40:51.617274 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.617263    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 00:40:51.617313 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.617275    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 00:40:51.617864 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.617832    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 00:40:51.617950 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.617887    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:51.717507 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.717484    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.718854 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.718786    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.718854 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.718823    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.718960 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.718861    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.718960 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.718927    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.722821 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.722765    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 00:40:51.724130 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.724116    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 00:40:51.724223 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.724136    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 00:40:51.724223 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.724154    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 00:40:51.724223 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.724182    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 00:40:51.740680 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.740662    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:40:51.741583 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.741569    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.741629 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.741593    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-17.ec2.internal\": node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:51.784924 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.784901    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:51.824923 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.824883    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"]
Oct 30 00:40:51.825059 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.824955    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.825935 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.825921    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.825987 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.825951    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.825987 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.825962    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.827755 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.827743    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.827921 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.827906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.827974 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.827941    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.828603 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.828587    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.828652 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.828621    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.828652 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.828632    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.828718 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.828587    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.828718 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.828689    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.828718 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.828707    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.830442 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.830423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.830442 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.830451    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:51.831298 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.831275    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:51.831372 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.831306    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:51.831372 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.831318    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:51.856528 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.856509    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-17.ec2.internal\" not found" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.859914 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.859899    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-17.ec2.internal\" not found" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.885105 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.885086    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:51.958743 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.958715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.958882 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.958748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.958882 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:51.958765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab07c2e2299f7ce9c9ccf7ec1f988b83-config\") pod \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" (UID: \"ab07c2e2299f7ce9c9ccf7ec1f988b83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:51.985912 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:51.985894    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.059887 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.059860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.060011 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.059890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.060011 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.059914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab07c2e2299f7ce9c9ccf7ec1f988b83-config\") pod \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" (UID: \"ab07c2e2299f7ce9c9ccf7ec1f988b83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.060011 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.059968    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.060011 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.060005    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab07c2e2299f7ce9c9ccf7ec1f988b83-config\") pod \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" (UID: \"ab07c2e2299f7ce9c9ccf7ec1f988b83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.060152 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.059975    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.086669 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.086642    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.160272 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.160245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.163191 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.163172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.186774 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.186751    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.287441 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.287348    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.387900 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.387868    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.488416 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.488385    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.493542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.493524    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 00:40:52.493697 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.493678    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 00:40:52.588913 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.588885    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.613441 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.613408    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 00:35:52 +0000 UTC" deadline="2028-03-25 18:19:59.933524451 +0000 UTC"
Oct 30 00:40:52.613441 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.613434    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12305h39m7.320093977s"
Oct 30 00:40:52.626692 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.626670    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:40:52.628386 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.628364    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:40:52.628480 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.628399    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:40:52.628480 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.628409    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:40:52.660694 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:52.660655    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab07c2e2299f7ce9c9ccf7ec1f988b83.slice/crio-0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7 WatchSource:0}: Error finding container 0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7: Status 404 returned error can't find the container with id 0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7
Oct 30 00:40:52.672084 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.672063    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:40:52.684634 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:52.684602    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd5d59005c843cb87cd033c834ec196.slice/crio-9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e WatchSource:0}: Error finding container 9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e: Status 404 returned error can't find the container with id 9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e
Oct 30 00:40:52.689139 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.689122    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.720593 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.720577    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:40:52.729747 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.729701    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" event={"ID":"0dd5d59005c843cb87cd033c834ec196","Type":"ContainerStarted","Data":"9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e"}
Oct 30 00:40:52.730787 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.730758    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal" event={"ID":"ab07c2e2299f7ce9c9ccf7ec1f988b83","Type":"ContainerStarted","Data":"0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7"}
Oct 30 00:40:52.789235 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:52.789214    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:40:52.815809 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.815762    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:40:52.856855 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.856830    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.881900 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.881880    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:40:52.884053 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.884040    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:40:52.905669 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:52.905647    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:40:53.126039 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.125951    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:40:53.533153 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.533127    2580 apiserver.go:52] "Watching apiserver"
Oct 30 00:40:53.557636 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.557604    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 00:40:53.566520 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.566490    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 00:40:53.566969 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.566938    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-clgnx","kube-system/konnectivity-agent-2ttqh","openshift-image-registry/node-ca-5q95w","openshift-multus/network-metrics-daemon-s2jdz","openshift-ovn-kubernetes/ovnkube-node-xp86c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4","openshift-network-diagnostics/network-check-target-qtsgh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal","openshift-multus/multus-additional-cni-plugins-ms722","openshift-multus/multus-dslp8","kube-system/global-pull-secret-syncer-7pfth","kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal","openshift-cluster-node-tuning-operator/tuned-57vkf","openshift-dns/node-resolver-pc8xn"]
Oct 30 00:40:53.569818 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.569794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:53.569965 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.569911    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:40:53.572353 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.572329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.575170 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.574736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.577166 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.577140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:53.577277 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.577243    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:40:53.580394 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.580373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.583299 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.583280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.587884 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.587041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.589694 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.589670    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.589783 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.589712    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-92qkq\""
Oct 30 00:40:53.589783 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.589753    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.589914 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.589898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.591606 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.591587    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 00:40:53.591723 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.591659    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.591985 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.591903    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.592662 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.592638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.594325 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.594308    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z5p7p\""
Oct 30 00:40:53.595850 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.595506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.598269 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.598140    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.602559 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.599259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.602559 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.599343    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:40:53.603696 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.603677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.622223 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.622156    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 00:40:53.622327 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.622227    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hlq82"
Oct 30 00:40:53.658294 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.658275    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 00:40:53.670867 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.670826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-bin\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671010 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.670895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-socket-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.671010 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.670918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-socket-dir-parent\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.671010 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.670944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f29d1b61-cd81-4277-af77-a44af9cc5237-tmp-dir\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.671010 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.670994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-conf\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-netns\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-var-lib-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-cnibin\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.671180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9c1e69b-d783-4731-8eba-61f21232cb31-konnectivity-ca\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-system-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29hz7\" (UniqueName: \"kubernetes.io/projected/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-kube-api-access-29hz7\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-config\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-kubernetes\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-lib-modules\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-ovn\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-device-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-run\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-bin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-registration-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7498e2f5-ea7b-4238-9111-bde1648f2fd1-host-slash\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-modprobe-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-kubelet\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-systemd-units\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-script-lib\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zp9v\" (UniqueName: \"kubernetes.io/projected/7498e2f5-ea7b-4238-9111-bde1648f2fd1-kube-api-access-7zp9v\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-kubelet-config\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-sys\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-node-log\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67jsn\" (UniqueName: \"kubernetes.io/projected/f50690df-0bf7-473a-a1e9-df93dacf3afa-kube-api-access-67jsn\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.671703 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-dbus\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f29d1b61-cd81-4277-af77-a44af9cc5237-hosts-file\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-var-lib-kubelet\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671760    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-slash\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-etc-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-env-overrides\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-tmp\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b995c294-397e-41e0-8a66-6e9b225ffd34-serviceca\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-netd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87cqm\" (UniqueName: \"kubernetes.io/projected/2e79761e-c9f1-494b-ba28-941db904b171-kube-api-access-87cqm\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-systemd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.671982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-multus\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysconfig\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-tuned\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-hostroot\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-etc-kubernetes\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.672235 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhjwm\" (UniqueName: \"kubernetes.io/projected/f29d1b61-cd81-4277-af77-a44af9cc5237-kube-api-access-dhjwm\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-host\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-sys-fs\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-netns\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-kubelet\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-log-socket\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbfacf74-0913-413c-b764-ec9afff26246-ovn-node-metrics-cert\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-system-cni-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4bqb\" (UniqueName: \"kubernetes.io/projected/7be17326-1cdc-43a1-b333-297b136611bf-kube-api-access-f4bqb\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9c1e69b-d783-4731-8eba-61f21232cb31-agent-certs\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-cnibin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673044 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-daemon-config\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m969p\" (UniqueName: \"kubernetes.io/projected/81f12500-d8ab-44de-bb4f-a5a987bddb69-kube-api-access-m969p\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b995c294-397e-41e0-8a66-6e9b225ffd34-host\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggq7r\" (UniqueName: \"kubernetes.io/projected/fbfacf74-0913-413c-b764-ec9afff26246-kube-api-access-ggq7r\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-cni-binary-copy\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-k8s-cni-cncf-io\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-multus-certs\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-systemd\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7498e2f5-ea7b-4238-9111-bde1648f2fd1-iptables-alerter-script\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-conf-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.673791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.672904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2927s\" (UniqueName: \"kubernetes.io/projected/b995c294-397e-41e0-8a66-6e9b225ffd34-kube-api-access-2927s\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.692953 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.692914    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:40:53.698253 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.698212    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 00:40:53.698401 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.698289    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 00:40:53.698401 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.698393    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.698675 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.698647    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 00:40:53.698989 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.698956    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.699618 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.699599    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.699733 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.699645    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.699815 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.699799    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7sn5q\""
Oct 30 00:40:53.699908 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.699861    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 00:40:53.700012 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.699990    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-cg6zp\""
Oct 30 00:40:53.700108 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700054    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.700108 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700100    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 00:40:53.700214 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700202    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-szv55\""
Oct 30 00:40:53.700323 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700305    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-f5gxs\""
Oct 30 00:40:53.700386 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700365    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7htjs\""
Oct 30 00:40:53.700433 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700388    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 00:40:53.700484 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700461    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 00:40:53.700567 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700551    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.700677 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700578    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 00:40:53.700677 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700640    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 00:40:53.700802 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700754    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 00:40:53.700894 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700853    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5b9kk\""
Oct 30 00:40:53.700952 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700911    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.701025 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.700995    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jgsn4\""
Oct 30 00:40:53.701114 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.701101    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 00:40:53.701180 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.701167    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 00:40:53.701248 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.701234    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 00:40:53.701306 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.701294    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.773926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9c1e69b-d783-4731-8eba-61f21232cb31-agent-certs\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.773975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-cnibin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-daemon-config\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m969p\" (UniqueName: \"kubernetes.io/projected/81f12500-d8ab-44de-bb4f-a5a987bddb69-kube-api-access-m969p\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b995c294-397e-41e0-8a66-6e9b225ffd34-host\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggq7r\" (UniqueName: \"kubernetes.io/projected/fbfacf74-0913-413c-b764-ec9afff26246-kube-api-access-ggq7r\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-cni-binary-copy\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-k8s-cni-cncf-io\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-multus-certs\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-systemd\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7498e2f5-ea7b-4238-9111-bde1648f2fd1-iptables-alerter-script\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-conf-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2927s\" (UniqueName: \"kubernetes.io/projected/b995c294-397e-41e0-8a66-6e9b225ffd34-kube-api-access-2927s\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.776842 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-bin\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-socket-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-socket-dir-parent\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f29d1b61-cd81-4277-af77-a44af9cc5237-tmp-dir\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-conf\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-netns\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-var-lib-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-cnibin\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9c1e69b-d783-4731-8eba-61f21232cb31-konnectivity-ca\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-system-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hz7\" (UniqueName: \"kubernetes.io/projected/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-kube-api-access-29hz7\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-config\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-kubernetes\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.777810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-lib-modules\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-ovn\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-device-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-run\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-bin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-registration-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7498e2f5-ea7b-4238-9111-bde1648f2fd1-host-slash\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-modprobe-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-kubelet\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-systemd-units\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-script-lib\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774893    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zp9v\" (UniqueName: \"kubernetes.io/projected/7498e2f5-ea7b-4238-9111-bde1648f2fd1-kube-api-access-7zp9v\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-kubelet-config\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.774995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-sys\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-node-log\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67jsn\" (UniqueName: \"kubernetes.io/projected/f50690df-0bf7-473a-a1e9-df93dacf3afa-kube-api-access-67jsn\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.778708 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-dbus\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f29d1b61-cd81-4277-af77-a44af9cc5237-hosts-file\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-var-lib-kubelet\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-slash\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775193    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-etc-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775258    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-etc-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-env-overrides\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775296    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-cnibin\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-tmp\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b995c294-397e-41e0-8a66-6e9b225ffd34-serviceca\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-netd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cqm\" (UniqueName: \"kubernetes.io/projected/2e79761e-c9f1-494b-ba28-941db904b171-kube-api-access-87cqm\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-systemd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-multus\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysconfig\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-tuned\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.779646 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-hostroot\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-etc-kubernetes\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhjwm\" (UniqueName: \"kubernetes.io/projected/f29d1b61-cd81-4277-af77-a44af9cc5237-kube-api-access-dhjwm\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-host\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775665    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-sys-fs\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-netns\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-kubelet\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-log-socket\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbfacf74-0913-413c-b764-ec9afff26246-ovn-node-metrics-cert\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-system-cni-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.775976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4bqb\" (UniqueName: \"kubernetes.io/projected/7be17326-1cdc-43a1-b333-297b136611bf-kube-api-access-f4bqb\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.780542 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.776950    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-system-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.777558    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9c1e69b-d783-4731-8eba-61f21232cb31-konnectivity-ca\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.777574    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-cnibin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.777744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.777865    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-kubernetes\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778006    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-lib-modules\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778063    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-ovn\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778118    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-device-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-run\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778234    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-bin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778291    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-registration-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778334    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7498e2f5-ea7b-4238-9111-bde1648f2fd1-host-slash\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778362    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-daemon-config\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778418    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-modprobe-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778467    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-kubelet\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778513    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-systemd-units\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b995c294-397e-41e0-8a66-6e9b225ffd34-host\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.778896    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.781436 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779125    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-config\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779464    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-cni-binary-copy\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779522    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-k8s-cni-cncf-io\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779553    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-multus-certs\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779595    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-systemd\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779670    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.779972    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-hostroot\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780029    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-etc-kubernetes\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780069    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7498e2f5-ea7b-4238-9111-bde1648f2fd1-iptables-alerter-script\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.780117    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780121    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-conf-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.780212    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret podName:f7088947-2eec-480b-9df3-285c4e6b2b59 nodeName:}" failed. No retries permitted until 2026-10-30 00:40:54.280173424 +0000 UTC m=+3.112612256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret") pod "global-pull-secret-syncer-7pfth" (UID: "f7088947-2eec-480b-9df3-285c4e6b2b59") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780382    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-bin\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780505    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-socket-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780521    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-host\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780571    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-socket-dir-parent\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780574    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.782370 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780663    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780719    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-sys-fs\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780805    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f29d1b61-cd81-4277-af77-a44af9cc5237-tmp-dir\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.780956    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781047    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-conf\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.781130    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.781177    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:40:54.28115879 +0000 UTC m=+3.113597597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781207    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-netns\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781246    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-var-lib-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781254    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781596    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-env-overrides\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781665    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-slash\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781709    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-kubelet-config\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781751    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-sys\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781796    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-node-log\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781804    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-netns\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.783064 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781860    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-kubelet\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781864    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-dbus\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.781907    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-log-socket\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.782006    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f29d1b61-cd81-4277-af77-a44af9cc5237-hosts-file\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.782059    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-var-lib-kubelet\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.782109    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-tuned\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.783096    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.783119    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-systemd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.783141    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-system-cni-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.783204    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-netd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.783874 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.783275    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-multus\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.785047 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.783325    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b995c294-397e-41e0-8a66-6e9b225ffd34-serviceca\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.785143 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.785080    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-tmp\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.785143 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.785116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbfacf74-0913-413c-b764-ec9afff26246-ovn-node-metrics-cert\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.785226 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.785158    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-script-lib\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.785263 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.785239    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysconfig\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.785359 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.785309    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.788412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.788372    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9c1e69b-d783-4731-8eba-61f21232cb31-agent-certs\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.802578 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.802548    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:40:53.802709 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.802586    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:40:53.802709 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.802602    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pgmpw for pod openshift-network-diagnostics/network-check-target-qtsgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:53.802709 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:53.802668    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw podName:1adb0772-ba0b-4eb7-9322-841e3ac21ebe nodeName:}" failed. No retries permitted until 2026-10-30 00:40:54.302651831 +0000 UTC m=+3.135090659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pgmpw" (UniqueName: "kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw") pod "network-check-target-qtsgh" (UID: "1adb0772-ba0b-4eb7-9322-841e3ac21ebe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:53.803185 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.803155    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4bqb\" (UniqueName: \"kubernetes.io/projected/7be17326-1cdc-43a1-b333-297b136611bf-kube-api-access-f4bqb\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.803810 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.803787    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29hz7\" (UniqueName: \"kubernetes.io/projected/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-kube-api-access-29hz7\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:53.805478 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.805413    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m969p\" (UniqueName: \"kubernetes.io/projected/81f12500-d8ab-44de-bb4f-a5a987bddb69-kube-api-access-m969p\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.805637 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.805614    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggq7r\" (UniqueName: \"kubernetes.io/projected/fbfacf74-0913-413c-b764-ec9afff26246-kube-api-access-ggq7r\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.808049 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.808013    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zp9v\" (UniqueName: \"kubernetes.io/projected/7498e2f5-ea7b-4238-9111-bde1648f2fd1-kube-api-access-7zp9v\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.808763 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.808733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2927s\" (UniqueName: \"kubernetes.io/projected/b995c294-397e-41e0-8a66-6e9b225ffd34-kube-api-access-2927s\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.813672 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.813645    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhjwm\" (UniqueName: \"kubernetes.io/projected/f29d1b61-cd81-4277-af77-a44af9cc5237-kube-api-access-dhjwm\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.814063 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.813914    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cqm\" (UniqueName: \"kubernetes.io/projected/2e79761e-c9f1-494b-ba28-941db904b171-kube-api-access-87cqm\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.814063 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.813939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67jsn\" (UniqueName: \"kubernetes.io/projected/f50690df-0bf7-473a-a1e9-df93dacf3afa-kube-api-access-67jsn\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.888412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.888375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:40:53.896751 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.896728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:40:53.908781 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.908752    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb995c294_397e_41e0_8a66_6e9b225ffd34.slice/crio-5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6 WatchSource:0}: Error finding container 5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6: Status 404 returned error can't find the container with id 5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6
Oct 30 00:40:53.927824 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.927796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:40:53.930007 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.929979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:40:53.932208 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.932182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:40:53.941027 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.940945    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbfacf74_0913_413c_b764_ec9afff26246.slice/crio-8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e WatchSource:0}: Error finding container 8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e: Status 404 returned error can't find the container with id 8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e
Oct 30 00:40:53.945093 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.945050    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e79761e_c9f1_494b_ba28_941db904b171.slice/crio-1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14 WatchSource:0}: Error finding container 1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14: Status 404 returned error can't find the container with id 1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14
Oct 30 00:40:53.945724 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.945537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:40:53.949235 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.949208    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29d1b61_cd81_4277_af77_a44af9cc5237.slice/crio-f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6 WatchSource:0}: Error finding container f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6: Status 404 returned error can't find the container with id f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6
Oct 30 00:40:53.951036 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.951014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:40:53.953902 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.953884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:40:53.956977 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:53.956771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dslp8"
Oct 30 00:40:53.960341 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.960313    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7498e2f5_ea7b_4238_9111_bde1648f2fd1.slice/crio-73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e WatchSource:0}: Error finding container 73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e: Status 404 returned error can't find the container with id 73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e
Oct 30 00:40:53.966608 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.966583    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be17326_1cdc_43a1_b333_297b136611bf.slice/crio-639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1 WatchSource:0}: Error finding container 639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1: Status 404 returned error can't find the container with id 639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1
Oct 30 00:40:53.972830 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:40:53.972803    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c1e69b_d783_4731_8eba_61f21232cb31.slice/crio-0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a WatchSource:0}: Error finding container 0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a: Status 404 returned error can't find the container with id 0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.381193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.381409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.381456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.381556    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.381602    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret podName:f7088947-2eec-480b-9df3-285c4e6b2b59 nodeName:}" failed. No retries permitted until 2026-10-30 00:40:55.381589784 +0000 UTC m=+4.214028591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret") pod "global-pull-secret-syncer-7pfth" (UID: "f7088947-2eec-480b-9df3-285c4e6b2b59") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.381891    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.381945    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:40:55.381932327 +0000 UTC m=+4.214371153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.382006    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.382016    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.382025    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pgmpw for pod openshift-network-diagnostics/network-check-target-qtsgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:54.382089 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:54.382049    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw podName:1adb0772-ba0b-4eb7-9322-841e3ac21ebe nodeName:}" failed. No retries permitted until 2026-10-30 00:40:55.382042406 +0000 UTC m=+4.214481218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgmpw" (UniqueName: "kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw") pod "network-check-target-qtsgh" (UID: "1adb0772-ba0b-4eb7-9322-841e3ac21ebe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:54.626193 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.626025    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 00:35:54 +0000 UTC" deadline="2028-03-25 09:43:27.393187906 +0000 UTC"
Oct 30 00:40:54.626193 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.626088    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12297h2m32.767103958s"
Oct 30 00:40:54.752730 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.752693    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-clgnx" event={"ID":"7498e2f5-ea7b-4238-9111-bde1648f2fd1","Type":"ContainerStarted","Data":"73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e"}
Oct 30 00:40:54.758801 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.758761    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pc8xn" event={"ID":"f29d1b61-cd81-4277-af77-a44af9cc5237","Type":"ContainerStarted","Data":"f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6"}
Oct 30 00:40:54.761210 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.761172    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e"}
Oct 30 00:40:54.771458 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.771425    2580 generic.go:356] "Generic (PLEG): container finished" podID="0dd5d59005c843cb87cd033c834ec196" containerID="bf19ad2be000c1ddfe2218b7a9c826fc982aa713d4260c223062d03837e293a6" exitCode=0
Oct 30 00:40:54.771593 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.771513    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" event={"ID":"0dd5d59005c843cb87cd033c834ec196","Type":"ContainerDied","Data":"bf19ad2be000c1ddfe2218b7a9c826fc982aa713d4260c223062d03837e293a6"}
Oct 30 00:40:54.782152 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.782121    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ttqh" event={"ID":"c9c1e69b-d783-4731-8eba-61f21232cb31","Type":"ContainerStarted","Data":"0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a"}
Oct 30 00:40:54.791127 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.791096    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4" event={"ID":"2e79761e-c9f1-494b-ba28-941db904b171","Type":"ContainerStarted","Data":"1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14"}
Oct 30 00:40:54.795669 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.795585    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5q95w" event={"ID":"b995c294-397e-41e0-8a66-6e9b225ffd34","Type":"ContainerStarted","Data":"5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6"}
Oct 30 00:40:54.823358 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.823123    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57vkf" event={"ID":"81f12500-d8ab-44de-bb4f-a5a987bddb69","Type":"ContainerStarted","Data":"e1336deeaf7e04ccbb32ceb87fece2a868fa57b766e9b25862670f4314b1ca36"}
Oct 30 00:40:54.831154 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.830487    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal" event={"ID":"ab07c2e2299f7ce9c9ccf7ec1f988b83","Type":"ContainerStarted","Data":"b060ed398719a24a973572837c91e0c1a0fa24fe5d9bc82bc767e31ff0b508ed"}
Oct 30 00:40:54.839381 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.839314    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dslp8" event={"ID":"f50690df-0bf7-473a-a1e9-df93dacf3afa","Type":"ContainerStarted","Data":"43c6311c7124ee6e289fe606e7eb3c45a47b8a5f1b4a68afb2f7f454bd49926e"}
Oct 30 00:40:54.844706 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:54.844625    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerStarted","Data":"639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1"}
Oct 30 00:40:55.390615 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.390584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:55.390615 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.390624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:55.390863 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.390649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:55.390863 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.390764    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:40:55.390863 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.390778    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:40:55.390863 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.390787    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pgmpw for pod openshift-network-diagnostics/network-check-target-qtsgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:55.390863 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.390830    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw podName:1adb0772-ba0b-4eb7-9322-841e3ac21ebe nodeName:}" failed. No retries permitted until 2026-10-30 00:40:57.390817229 +0000 UTC m=+6.223256050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgmpw" (UniqueName: "kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw") pod "network-check-target-qtsgh" (UID: "1adb0772-ba0b-4eb7-9322-841e3ac21ebe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:55.391187 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.391170    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:55.391252 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.391216    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret podName:f7088947-2eec-480b-9df3-285c4e6b2b59 nodeName:}" failed. No retries permitted until 2026-10-30 00:40:57.391205305 +0000 UTC m=+6.223644127 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret") pod "global-pull-secret-syncer-7pfth" (UID: "f7088947-2eec-480b-9df3-285c4e6b2b59") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:55.391317 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.391254    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:55.391317 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.391275    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:40:57.391268019 +0000 UTC m=+6.223706826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:55.626624 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.626573    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 00:35:54 +0000 UTC" deadline="2028-07-10 14:23:20.819516214 +0000 UTC"
Oct 30 00:40:55.626624 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.626622    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14869h42m25.192899639s"
Oct 30 00:40:55.733056 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.733023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:55.733240 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.733163    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:40:55.733628 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.733606    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:55.733752 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.733726    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:40:55.733817 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.733807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:55.733911 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:55.733892    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:40:55.864566 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.864527    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" event={"ID":"0dd5d59005c843cb87cd033c834ec196","Type":"ContainerStarted","Data":"7764e08df45ea69c98765b7e67c46e0af987f79dd8c9aff8d04f528187d08286"}
Oct 30 00:40:55.913142 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:55.911763    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal" podStartSLOduration=2.911740433 podStartE2EDuration="2.911740433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:40:54.87639681 +0000 UTC m=+3.708835643" watchObservedRunningTime="2026-10-30 00:40:55.911740433 +0000 UTC m=+4.744179263"
Oct 30 00:40:56.874663 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:56.874579    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerStarted","Data":"4cf698abb7fd8da55955dd9d0b0eeba19c8a375c918f5224ab340daff1be76d0"}
Oct 30 00:40:56.956487 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:56.956420    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" podStartSLOduration=3.956398821 podStartE2EDuration="3.956398821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:40:55.9131246 +0000 UTC m=+4.745563433" watchObservedRunningTime="2026-10-30 00:40:56.956398821 +0000 UTC m=+5.788837651"
Oct 30 00:40:57.410860 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.410822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:57.411057 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.410913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:57.411057 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.410948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:57.411131 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411094    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:40:57.411131 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411112    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:40:57.411213 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411202    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pgmpw for pod openshift-network-diagnostics/network-check-target-qtsgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:57.411283 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411269    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw podName:1adb0772-ba0b-4eb7-9322-841e3ac21ebe nodeName:}" failed. No retries permitted until 2026-10-30 00:41:01.411250575 +0000 UTC m=+10.243689397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgmpw" (UniqueName: "kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw") pod "network-check-target-qtsgh" (UID: "1adb0772-ba0b-4eb7-9322-841e3ac21ebe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:40:57.411697 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411679    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:57.411748 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411737    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret podName:f7088947-2eec-480b-9df3-285c4e6b2b59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:01.411720081 +0000 UTC m=+10.244158891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret") pod "global-pull-secret-syncer-7pfth" (UID: "f7088947-2eec-480b-9df3-285c4e6b2b59") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:40:57.411816 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411804    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:57.411851 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.411842    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:01.411830906 +0000 UTC m=+10.244269717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:40:57.724762 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.724681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:57.724937 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.724814    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:40:57.725282 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.725250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:57.725421 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.725368    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:40:57.725498 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.725445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:57.725559 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:57.725545    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:40:57.878512 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.878473    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="4cf698abb7fd8da55955dd9d0b0eeba19c8a375c918f5224ab340daff1be76d0" exitCode=0
Oct 30 00:40:57.879075 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:57.878525    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerDied","Data":"4cf698abb7fd8da55955dd9d0b0eeba19c8a375c918f5224ab340daff1be76d0"}
Oct 30 00:40:58.888365 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:58.888317    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ttqh" event={"ID":"c9c1e69b-d783-4731-8eba-61f21232cb31","Type":"ContainerStarted","Data":"12f6314e7781d42a7e5ee96b0114ff4ebf6575804cae3a6defa5478e27557684"}
Oct 30 00:40:59.731266 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:59.731229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:40:59.731498 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:59.731377    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:40:59.731875 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:59.731837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:40:59.732011 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:59.731986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:40:59.732080 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:59.732058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:40:59.732173 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:40:59.732150    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:40:59.896628 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:59.896593    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5q95w" event={"ID":"b995c294-397e-41e0-8a66-6e9b225ffd34","Type":"ContainerStarted","Data":"6dd612ef60d27fa0d08c040a5cbba64e58b99bae8cb8fc85d047362ec9bc031e"}
Oct 30 00:40:59.941373 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:59.941306    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2ttqh" podStartSLOduration=3.9554772910000002 podStartE2EDuration="7.94128397s" totalImagesPullingTime="3.985806679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:40:58.936422448 +0000 UTC m=+7.768861282" watchObservedRunningTime="2026-10-30 00:40:59.94128397 +0000 UTC m=+8.773722802"
Oct 30 00:40:59.941564 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:40:59.941424    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5q95w" podStartSLOduration=3.072017062 podStartE2EDuration="7.941414936s" totalImagesPullingTime="4.869397874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:40:59.940834757 +0000 UTC m=+8.773273590" watchObservedRunningTime="2026-10-30 00:40:59.941414936 +0000 UTC m=+8.773853767"
Oct 30 00:41:00.904122 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:00.904037    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4" event={"ID":"2e79761e-c9f1-494b-ba28-941db904b171","Type":"ContainerStarted","Data":"0b6c29df30995b42fc3408b2a9c63f9bdc2a84ee29322db9532c69a9b2818e5a"}
Oct 30 00:41:01.446245 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.446211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:01.446416 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.446256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:01.446416 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.446300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:01.446416 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.446411    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:41:01.446599 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.446485    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret podName:f7088947-2eec-480b-9df3-285c4e6b2b59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:09.446455598 +0000 UTC m=+18.278894405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret") pod "global-pull-secret-syncer-7pfth" (UID: "f7088947-2eec-480b-9df3-285c4e6b2b59") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:41:01.446936 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.446915    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:41:01.447044 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.446976    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:09.446960318 +0000 UTC m=+18.279399137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:41:01.447121 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.447057    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:41:01.447121 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.447075    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:41:01.447121 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.447087    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pgmpw for pod openshift-network-diagnostics/network-check-target-qtsgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:41:01.447265 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.447135    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw podName:1adb0772-ba0b-4eb7-9322-841e3ac21ebe nodeName:}" failed. No retries permitted until 2026-10-30 00:41:09.44711203 +0000 UTC m=+18.279550849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgmpw" (UniqueName: "kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw") pod "network-check-target-qtsgh" (UID: "1adb0772-ba0b-4eb7-9322-841e3ac21ebe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:41:01.725095 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.725066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:01.725219 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.725195    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:01.725553 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.725536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:01.725642 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.725624    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:01.725694 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.725680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:01.725748 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:01.725739    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:01.832401 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.832371    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 00:41:01.916870 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.916819    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4" event={"ID":"2e79761e-c9f1-494b-ba28-941db904b171","Type":"ContainerStarted","Data":"c53606920909d0d476b402518baab3af7e186c5bface89acfc537c9f356915d7"}
Oct 30 00:41:01.919323 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:01.919283    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dslp8" event={"ID":"f50690df-0bf7-473a-a1e9-df93dacf3afa","Type":"ContainerStarted","Data":"aae8683b577c8e1d2aee21bb1387aa97b2e35cbe77591ad613afdf6eac70afa9"}
Oct 30 00:41:02.122281 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.122006    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:02.122628 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.122598    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:02.166052 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.165998    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-dslp8" podStartSLOduration=2.465016223 podStartE2EDuration="10.165983927s" totalImagesPullingTime="7.700967704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:01.965612083 +0000 UTC m=+10.798050910" watchObservedRunningTime="2026-10-30 00:41:02.165983927 +0000 UTC m=+10.998422751"
Oct 30 00:41:02.621907 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.621741    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T00:41:01.832393177Z","UUID":"502d5f13-59a2-46dc-9fd3-5d1ce4e44222","Handler":null,"Name":"","Endpoint":""}
Oct 30 00:41:02.623322 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.623285    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 00:41:02.623322 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.623318    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 00:41:02.925302 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.925265    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="d792de6a3f23cfc7bf85ebcf54b8a22022593b388a78cd2ce05ca17ea2cdab4b" exitCode=0
Oct 30 00:41:02.925808 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.925310    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerDied","Data":"d792de6a3f23cfc7bf85ebcf54b8a22022593b388a78cd2ce05ca17ea2cdab4b"}
Oct 30 00:41:02.928589 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.928478    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pc8xn" event={"ID":"f29d1b61-cd81-4277-af77-a44af9cc5237","Type":"ContainerStarted","Data":"fbef8b16c211bac85f0a45dc49d020cabc4fbc8dc615ad03c7a04bacfaabe2fc"}
Oct 30 00:41:02.929091 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.929073    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:02.929730 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:02.929701    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:03.725357 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.725324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:03.725542 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:03.725464    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:03.725819 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.725801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:03.725935 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:03.725908    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:03.726056 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.726003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:03.726127 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:03.726061    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:03.933709 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.933113    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-clgnx" event={"ID":"7498e2f5-ea7b-4238-9111-bde1648f2fd1","Type":"ContainerStarted","Data":"f5bc800503e01eca1bce7e278968f1bb3bfe4f372def035041b1c584a3e996e5"}
Oct 30 00:41:03.935958 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.935880    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4" event={"ID":"2e79761e-c9f1-494b-ba28-941db904b171","Type":"ContainerStarted","Data":"0b2d047135438a542f33ea375fbad68d558d7d8e382d0d53db48f1b56eeaf9bb"}
Oct 30 00:41:03.974501 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.974446    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-pc8xn" podStartSLOduration=3.630380925 podStartE2EDuration="11.97442662s" totalImagesPullingTime="8.344045695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:03.060525962 +0000 UTC m=+11.892964791" watchObservedRunningTime="2026-10-30 00:41:03.97442662 +0000 UTC m=+12.806865443"
Oct 30 00:41:03.974656 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:03.974577    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-clgnx" podStartSLOduration=3.955218455 podStartE2EDuration="11.974569979s" totalImagesPullingTime="8.019351524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:03.974397243 +0000 UTC m=+12.806836073" watchObservedRunningTime="2026-10-30 00:41:03.974569979 +0000 UTC m=+12.807008806"
Oct 30 00:41:04.019422 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:04.019312    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4" podStartSLOduration=3.448128693 podStartE2EDuration="12.019289849s" totalImagesPullingTime="8.571161156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:04.018324041 +0000 UTC m=+12.850762869" watchObservedRunningTime="2026-10-30 00:41:04.019289849 +0000 UTC m=+12.851728677"
Oct 30 00:41:04.940835 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:04.940800    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="27722fae50d12a818c8678871719dedd2966401791db522efe2c94a341d1285d" exitCode=0
Oct 30 00:41:04.941580 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:04.940884    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerDied","Data":"27722fae50d12a818c8678871719dedd2966401791db522efe2c94a341d1285d"}
Oct 30 00:41:04.943790 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:04.943713    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57vkf" event={"ID":"81f12500-d8ab-44de-bb4f-a5a987bddb69","Type":"ContainerStarted","Data":"cbe0d070c48cb59d096535548f75a1662f81a7ebd511f075d3c7655e20acaf56"}
Oct 30 00:41:05.724510 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:05.724478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:05.724675 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:05.724551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:05.724675 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:05.724554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:05.724767 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:05.724665    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:05.724767 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:05.724727    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:05.724893 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:05.724823    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:06.951988 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:06.951722    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="e1f70b3a3420080b9e8724191834b21b63db3dcbce4b60cd7d463e211d67ab5f" exitCode=0
Oct 30 00:41:06.951988 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:06.951800    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerDied","Data":"e1f70b3a3420080b9e8724191834b21b63db3dcbce4b60cd7d463e211d67ab5f"}
Oct 30 00:41:07.028639 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:07.028588    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-57vkf" podStartSLOduration=4.485412516 podStartE2EDuration="15.028575295s" totalImagesPullingTime="10.543162779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:05.048817576 +0000 UTC m=+13.881256404" watchObservedRunningTime="2026-10-30 00:41:07.028575295 +0000 UTC m=+15.861014121"
Oct 30 00:41:07.724833 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:07.724791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:07.724833 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:07.724856    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:07.725100 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:07.724972    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:07.725429 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:07.725234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:07.725429 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:07.725377    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:07.725601 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:07.725460    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:08.959599 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:08.959570    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerStarted","Data":"c1807d0197c1150ab6d6fb0899f8003a84781ea8ecfef1d745866098d79e3683"}
Oct 30 00:41:08.961205 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:08.961170    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"c39a54385b34806aecefde771993aa962ece5662885af1937748fff0e2652cf9"}
Oct 30 00:41:09.509606 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.509431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.509636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.509658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509585    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509760    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509770    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509784    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:41:09.509788 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509793    2580 projected.go:196] Error preparing data for projected volume kube-api-access-pgmpw for pod openshift-network-diagnostics/network-check-target-qtsgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:41:09.510011 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509767    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret podName:f7088947-2eec-480b-9df3-285c4e6b2b59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:25.509745218 +0000 UTC m=+34.342184027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret") pod "global-pull-secret-syncer-7pfth" (UID: "f7088947-2eec-480b-9df3-285c4e6b2b59") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:41:09.510011 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509862    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:25.509825027 +0000 UTC m=+34.342263836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:41:09.510011 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.509879    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw podName:1adb0772-ba0b-4eb7-9322-841e3ac21ebe nodeName:}" failed. No retries permitted until 2026-10-30 00:41:25.509870383 +0000 UTC m=+34.342309188 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgmpw" (UniqueName: "kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw") pod "network-check-target-qtsgh" (UID: "1adb0772-ba0b-4eb7-9322-841e3ac21ebe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:41:09.724706 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.724667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:09.724706 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.724688    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:09.724953 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.724720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:09.724953 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.724825    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:09.725073 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.724922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:09.725073 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:09.724995    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:09.965409 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.965324    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="c1807d0197c1150ab6d6fb0899f8003a84781ea8ecfef1d745866098d79e3683" exitCode=0
Oct 30 00:41:09.965409 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.965399    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerDied","Data":"c1807d0197c1150ab6d6fb0899f8003a84781ea8ecfef1d745866098d79e3683"}
Oct 30 00:41:09.968550 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.968531    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"b7c929860330c42ee0c34cf700e6ae96107d5e9926fdf76cab3ce57643a84261"}
Oct 30 00:41:09.968654 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.968587    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"b873b448a055b6527dfbffb4601e43faae278d688af1d71b6ad53684beb1a1b6"}
Oct 30 00:41:09.968654 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.968616    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"3eaa0c59face333934bcaadb940ad3e87d4172671115c5298759a7fac1ff587f"}
Oct 30 00:41:09.968654 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.968652    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"1dfcad842efd38e1d7d6934ce19b1c8a65beee72740318a97f17a09d51a71d52"}
Oct 30 00:41:09.968804 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:09.968678    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"b8175d14a3c3ab74e22e98c2185e6352e36d24c0103c29364098b7602bfb1b13"}
Oct 30 00:41:10.973822 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:10.973790    2580 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="030d8eed4afdb444c8d8f7589acb229cf86d68ad1d0e65a876f73f652f78ef9f" exitCode=0
Oct 30 00:41:10.974315 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:10.973870    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerDied","Data":"030d8eed4afdb444c8d8f7589acb229cf86d68ad1d0e65a876f73f652f78ef9f"}
Oct 30 00:41:11.725706 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:11.725627    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:11.725872 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:11.725727    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:11.725872 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:11.725735    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:11.725872 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:11.725818    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:11.725872 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:11.725833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:11.726012 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:11.725989    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:11.978996 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:11.978961    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ms722" event={"ID":"7be17326-1cdc-43a1-b333-297b136611bf","Type":"ContainerStarted","Data":"660538b7ad2a07afefd88e0d2f8787da024071ea9122e0dd079612d253424a30"}
Oct 30 00:41:11.982157 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:11.982118    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"d6613bfc008e5c8f533e90b49948a7dc15ea264402abecafed65f61a291c8bd5"}
Oct 30 00:41:12.054716 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:12.054663    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ms722" podStartSLOduration=9.21527112 podStartE2EDuration="20.0546504s" totalImagesPullingTime="10.83937928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:12.054277652 +0000 UTC m=+20.886716479" watchObservedRunningTime="2026-10-30 00:41:12.0546504 +0000 UTC m=+20.887089224"
Oct 30 00:41:13.725619 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:13.725359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:13.725619 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:13.725384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:13.726094 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:13.725616    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:13.726094 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:13.725428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:13.726094 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:13.725691    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:13.726094 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:13.725805    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:13.991106 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:13.991077    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" event={"ID":"fbfacf74-0913-413c-b764-ec9afff26246","Type":"ContainerStarted","Data":"28e549593eeef9e8e696b536626e61dc08a7d700731c04f1f74a874d317980aa"}
Oct 30 00:41:13.991350 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:13.991331    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:14.007727 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:14.007703    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:14.084750 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:14.084697    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c" podStartSLOduration=7.156561805 podStartE2EDuration="22.084679318s" totalImagesPullingTime="14.928117513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:14.071177427 +0000 UTC m=+22.903616253" watchObservedRunningTime="2026-10-30 00:41:14.084679318 +0000 UTC m=+22.917118145"
Oct 30 00:41:14.993001 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:14.992973    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:41:14.993466 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:14.993399    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:15.009002 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.008983    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:15.404355 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.404278    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7pfth"]
Oct 30 00:41:15.404592 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.404465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:15.404592 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:15.404565    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:15.407753 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.407725    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtsgh"]
Oct 30 00:41:15.407899 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.407886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:15.407985 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:15.407970    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:15.408384 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.408361    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s2jdz"]
Oct 30 00:41:15.408491 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.408479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:15.408615 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:15.408594    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:15.995356 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:15.995329    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:41:16.724835 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:16.724803    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:16.725005 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:16.724858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:16.725005 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:16.724935    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:16.725088 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:16.725060    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:16.997291 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:16.997264    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:41:17.647902 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:17.647868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:17.725162 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:17.725087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:17.725320 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:17.725196    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:17.812649 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:17.812617    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:18.551386 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:18.551308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:18.661998 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:18.661969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:18.724534 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:18.724507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:18.724666 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:18.724613    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:18.724705 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:18.724665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:18.724760 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:18.724745    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:19.725383 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:19.725352    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:19.725779 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:19.725451    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:20.399264 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.399186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:20.510123 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.510094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:20.724945 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.724871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:20.724945 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.724882    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:20.725128 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:20.724998    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s2jdz" podUID="d2907a2d-9dbd-4dae-93d6-841dfcd2a596"
Oct 30 00:41:20.725174 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:20.725127    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7pfth" podUID="f7088947-2eec-480b-9df3-285c4e6b2b59"
Oct 30 00:41:20.849457 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.849432    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:20.849898 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.849655    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:41:20.861238 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:20.861216    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:21.086451 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.086416    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:21.202933 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.202905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:21.372930 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.372837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:21.506372 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.506341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:21.656362 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.656149    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:21.726027 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.725996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:21.726194 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:21.726100    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtsgh" podUID="1adb0772-ba0b-4eb7-9322-841e3ac21ebe"
Oct 30 00:41:21.823868 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.823810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:21.933013 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:21.932942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:22.454657 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.454631    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeReady"
Oct 30 00:41:22.454820 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.454757    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 00:41:22.516331 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.516302    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hmkqx"]
Oct 30 00:41:22.518419 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.518404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.524940 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.524916    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2g945"]
Oct 30 00:41:22.526719 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.526705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.540149 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.540128    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 00:41:22.540909 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.540892    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hmkqx"]
Oct 30 00:41:22.541359 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.541345    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 00:41:22.541437 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.541407    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l8pvs\""
Oct 30 00:41:22.546739 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.546719    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2g945"]
Oct 30 00:41:22.601661 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.601635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-config-volume\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.601661 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.601660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.601813 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.601679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5ws\" (UniqueName: \"kubernetes.io/projected/06eb66bd-1808-4c71-9a97-d3da60d23201-kube-api-access-xf5ws\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.601813 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.601698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-tmp-dir\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.601813 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.601782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzc66\" (UniqueName: \"kubernetes.io/projected/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-kube-api-access-tzc66\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.601931 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.601838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06eb66bd-1808-4c71-9a97-d3da60d23201-cert\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.641534 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.641504    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-x7m5x\""
Oct 30 00:41:22.641534 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.641530    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 00:41:22.641689 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.641567    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 00:41:22.641689 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.641584    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 00:41:22.703054 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-config-volume\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.703163 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.703163 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5ws\" (UniqueName: \"kubernetes.io/projected/06eb66bd-1808-4c71-9a97-d3da60d23201-kube-api-access-xf5ws\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.703163 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-tmp-dir\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.703163 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzc66\" (UniqueName: \"kubernetes.io/projected/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-kube-api-access-tzc66\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.703342 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:22.703212    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 00:41:22.703342 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06eb66bd-1808-4c71-9a97-d3da60d23201-cert\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.703342 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:22.703277    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls podName:ac84b9b8-dee8-4ecf-acd0-6b093753b60c nodeName:}" failed. No retries permitted until 2026-10-30 00:41:23.203255653 +0000 UTC m=+32.035694471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls") pod "dns-default-hmkqx" (UID: "ac84b9b8-dee8-4ecf-acd0-6b093753b60c") : secret "dns-default-metrics-tls" not found
Oct 30 00:41:22.703506 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703490    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-tmp-dir\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.703718 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.703689    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-config-volume\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.707265 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.707228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06eb66bd-1808-4c71-9a97-d3da60d23201-cert\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.724372 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.724357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:22.724500 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.724357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:22.727911 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.727883    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzc66\" (UniqueName: \"kubernetes.io/projected/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-kube-api-access-tzc66\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:22.728018 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.727907    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5ws\" (UniqueName: \"kubernetes.io/projected/06eb66bd-1808-4c71-9a97-d3da60d23201-kube-api-access-xf5ws\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.741622 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.741590    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 00:41:22.742040 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.742021    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wwjm6\""
Oct 30 00:41:22.742040 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.742039    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z4qxk\""
Oct 30 00:41:22.742146 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.742044    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 00:41:22.835277 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.835250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:22.948159 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.948131    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2v4fv"]
Oct 30 00:41:22.950103 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.950088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:22.967554 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.967510    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 00:41:22.967554 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.967548    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 00:41:22.970629 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.970615    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 00:41:22.970881 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.970866    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 00:41:22.971639 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.971621    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qs9gb\""
Oct 30 00:41:22.971985 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:22.971970    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2v4fv"]
Oct 30 00:41:23.018112 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.018090    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-blf84-qzhkv"]
Oct 30 00:41:23.021687 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.021669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.021984 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.021963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:23.039974 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.039949    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-blf84-qzhkv"]
Oct 30 00:41:23.067861 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.067828    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 00:41:23.070912 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.068744    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-748hz\""
Oct 30 00:41:23.082093 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.082070    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 00:41:23.106319 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.106433 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ad924ba-4fd0-4a90-b699-01c066c8de59-crio-socket\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.106492 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.106536 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvl8d\" (UniqueName: \"kubernetes.io/projected/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-api-access-vvl8d\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.106573 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.106573 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.106657 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.106657 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.106657 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.106639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad924ba-4fd0-4a90-b699-01c066c8de59-data-volume\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.134666 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.134644    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2g945"]
Oct 30 00:41:23.138204 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:23.138182    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06eb66bd_1808_4c71_9a97_d3da60d23201.slice/crio-81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a WatchSource:0}: Error finding container 81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a: Status 404 returned error can't find the container with id 81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a
Oct 30 00:41:23.142331 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.142303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:23.207475 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ad924ba-4fd0-4a90-b699-01c066c8de59-crio-socket\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.207577 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:23.207614 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.207583    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 00:41:23.207645 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.207645 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.207627    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls podName:ac84b9b8-dee8-4ecf-acd0-6b093753b60c nodeName:}" failed. No retries permitted until 2026-10-30 00:41:24.207615036 +0000 UTC m=+33.040053843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls") pod "dns-default-hmkqx" (UID: "ac84b9b8-dee8-4ecf-acd0-6b093753b60c") : secret "dns-default-metrics-tls" not found
Oct 30 00:41:23.207713 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207640    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ad924ba-4fd0-4a90-b699-01c066c8de59-crio-socket\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.207713 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvl8d\" (UniqueName: \"kubernetes.io/projected/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-api-access-vvl8d\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.207713 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.207677    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:23.207811 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.207811 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.207720    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:23.707707818 +0000 UTC m=+32.540146622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:23.207811 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.207811 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.207811 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.208068 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.207905    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 00:41:23.208068 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.207900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad924ba-4fd0-4a90-b699-01c066c8de59-data-volume\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.208068 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.207973    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:23.707954076 +0000 UTC m=+32.540392899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : configmap "kube-rbac-proxy" not found
Oct 30 00:41:23.208068 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.208046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.208259 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.208218    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad924ba-4fd0-4a90-b699-01c066c8de59-data-volume\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.208315 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.208307    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.208395 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.208376    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.208657 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.208640    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.231758 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.231733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvl8d\" (UniqueName: \"kubernetes.io/projected/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-api-access-vvl8d\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.231840 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.231787    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.331605 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.331573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:23.333207 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.333186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:23.367474 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.367443    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"]
Oct 30 00:41:23.369623 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.369602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:23.389949 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.389770    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 00:41:23.390791 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.390766    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"]
Oct 30 00:41:23.391034 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.391011    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kdklf\""
Oct 30 00:41:23.391147 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.391111    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 00:41:23.485178 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.485103    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:23.510341 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.510309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klrps\" (UniqueName: \"kubernetes.io/projected/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf-kube-api-access-klrps\") pod \"migrator-66df5d496b-k6f7p\" (UID: \"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:23.608567 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.608536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:23.611517 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.611463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klrps\" (UniqueName: \"kubernetes.io/projected/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf-kube-api-access-klrps\") pod \"migrator-66df5d496b-k6f7p\" (UID: \"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:23.624960 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.624932    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-blf84-qzhkv"]
Oct 30 00:41:23.627718 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:23.627695    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490ed73e_84fe_4053_bec4_43f479935b3d.slice/crio-56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684 WatchSource:0}: Error finding container 56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684: Status 404 returned error can't find the container with id 56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684
Oct 30 00:41:23.636554 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.636530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klrps\" (UniqueName: \"kubernetes.io/projected/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf-kube-api-access-klrps\") pod \"migrator-66df5d496b-k6f7p\" (UID: \"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:23.681816 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.681789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:23.712832 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.712802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.713002 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.712874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:23.713002 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.712958    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:23.713002 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.713002    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 00:41:23.713120 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.713019    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:24.713002788 +0000 UTC m=+33.545441609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:23.713120 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:23.713036    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:24.713027226 +0000 UTC m=+33.545466030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : configmap "kube-rbac-proxy" not found
Oct 30 00:41:23.724922 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.724899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:23.741417 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.741396    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-wm8np\""
Oct 30 00:41:23.741501 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.741400    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 00:41:23.741501 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.741427    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 00:41:23.928561 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.928528    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-dpp79"]
Oct 30 00:41:23.932742 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.932716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:23.949481 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.949456    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 00:41:23.949481 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.949478    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 00:41:23.949833 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.949456    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 00:41:23.949833 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.949536    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 00:41:23.949833 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.949538    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qtb46\""
Oct 30 00:41:23.950330 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.950312    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-dpp79"]
Oct 30 00:41:23.981218 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:23.981190    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"]
Oct 30 00:41:23.984581 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:23.984558    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e80bbb_3f65_4513_8fe3_8c43c2fa83bf.slice/crio-1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9 WatchSource:0}: Error finding container 1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9: Status 404 returned error can't find the container with id 1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9
Oct 30 00:41:24.013732 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.013649    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p" event={"ID":"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf","Type":"ContainerStarted","Data":"1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9"}
Oct 30 00:41:24.015056 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.015029    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-blf84-qzhkv" event={"ID":"490ed73e-84fe-4053-bec4-43f479935b3d","Type":"ContainerStarted","Data":"56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684"}
Oct 30 00:41:24.015688 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.015671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-cabundle\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.015770 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.015700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-key\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.015929 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.015907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg9gh\" (UniqueName: \"kubernetes.io/projected/87db4abb-36f5-4135-ae3a-f03cb9e2d786-kube-api-access-sg9gh\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.016315 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.016294    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2g945" event={"ID":"06eb66bd-1808-4c71-9a97-d3da60d23201","Type":"ContainerStarted","Data":"81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a"}
Oct 30 00:41:24.116564 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.116533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg9gh\" (UniqueName: \"kubernetes.io/projected/87db4abb-36f5-4135-ae3a-f03cb9e2d786-kube-api-access-sg9gh\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.116745 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.116591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-cabundle\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.116745 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.116619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-key\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.117667 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.117636    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-cabundle\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.119566 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.119542    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-key\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.142675 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.142640    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg9gh\" (UniqueName: \"kubernetes.io/projected/87db4abb-36f5-4135-ae3a-f03cb9e2d786-kube-api-access-sg9gh\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.217612 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.217581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:24.217772 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:24.217752    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 00:41:24.217875 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:24.217835    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls podName:ac84b9b8-dee8-4ecf-acd0-6b093753b60c nodeName:}" failed. No retries permitted until 2026-10-30 00:41:26.217810679 +0000 UTC m=+35.050249488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls") pod "dns-default-hmkqx" (UID: "ac84b9b8-dee8-4ecf-acd0-6b093753b60c") : secret "dns-default-metrics-tls" not found
Oct 30 00:41:24.246193 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.246159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:24.354485 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.354448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:24.482650 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.482612    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:24.547220 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.547181    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-dpp79"]
Oct 30 00:41:24.552533 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:24.552495    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87db4abb_36f5_4135_ae3a_f03cb9e2d786.slice/crio-fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf WatchSource:0}: Error finding container fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf: Status 404 returned error can't find the container with id fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf
Oct 30 00:41:24.724871 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.724824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:24.725009 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.724904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:24.725009 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:24.724965    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:24.725009 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:24.724995    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 00:41:24.725123 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:24.725024    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:26.725006693 +0000 UTC m=+35.557445501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:24.725123 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:24.725042    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:26.725033321 +0000 UTC m=+35.557472133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : configmap "kube-rbac-proxy" not found
Oct 30 00:41:24.865184 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:24.865067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:25.023982 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.023948    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79" event={"ID":"87db4abb-36f5-4135-ae3a-f03cb9e2d786","Type":"ContainerStarted","Data":"fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf"}
Oct 30 00:41:25.530638 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.530588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:25.530817 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.530676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:25.530817 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.530704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:25.531421 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:25.531108    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 00:41:25.531421 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:25.531193    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs podName:d2907a2d-9dbd-4dae-93d6-841dfcd2a596 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:57.531172109 +0000 UTC m=+66.363610927 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs") pod "network-metrics-daemon-s2jdz" (UID: "d2907a2d-9dbd-4dae-93d6-841dfcd2a596") : secret "metrics-daemon-secret" not found
Oct 30 00:41:25.534122 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.534100    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:25.534552 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.534533    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:25.545107 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.545084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:25.740697 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.740664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:25.875177 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:25.874424    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtsgh"]
Oct 30 00:41:25.878740 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:25.878710    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1adb0772_ba0b_4eb7_9322_841e3ac21ebe.slice/crio-73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3 WatchSource:0}: Error finding container 73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3: Status 404 returned error can't find the container with id 73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3
Oct 30 00:41:26.028924 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.028781    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtsgh" event={"ID":"1adb0772-ba0b-4eb7-9322-841e3ac21ebe","Type":"ContainerStarted","Data":"73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3"}
Oct 30 00:41:26.030927 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.030836    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p" event={"ID":"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf","Type":"ContainerStarted","Data":"198c8a4b59ad088a74fa86c4a87e2a0838b5367c3cbfc7e373578106cd58c097"}
Oct 30 00:41:26.060404 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.060375    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7pfth"]
Oct 30 00:41:26.063940 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:26.063905    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7088947_2eec_480b_9df3_285c4e6b2b59.slice/crio-749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867 WatchSource:0}: Error finding container 749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867: Status 404 returned error can't find the container with id 749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867
Oct 30 00:41:26.073623 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.073584    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p" podStartSLOduration=1.7966286070000002 podStartE2EDuration="3.073572551s" totalImagesPullingTime="1.276943944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:26.072990601 +0000 UTC m=+34.905429428" watchObservedRunningTime="2026-10-30 00:41:26.073572551 +0000 UTC m=+34.906011372"
Oct 30 00:41:26.237039 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.237004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:26.237235 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:26.237210    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 00:41:26.237381 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:26.237309    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls podName:ac84b9b8-dee8-4ecf-acd0-6b093753b60c nodeName:}" failed. No retries permitted until 2026-10-30 00:41:30.237272261 +0000 UTC m=+39.069711078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls") pod "dns-default-hmkqx" (UID: "ac84b9b8-dee8-4ecf-acd0-6b093753b60c") : secret "dns-default-metrics-tls" not found
Oct 30 00:41:26.741895 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.741840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:26.742050 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:26.741920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:26.742137 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:26.742053    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:26.742137 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:26.742061    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 00:41:26.742137 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:26.742137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:30.742113568 +0000 UTC m=+39.574552388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:26.742317 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:26.742157    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:30.742147659 +0000 UTC m=+39.574586463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : configmap "kube-rbac-proxy" not found
Oct 30 00:41:27.035896 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:27.035838    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-blf84-qzhkv" event={"ID":"490ed73e-84fe-4053-bec4-43f479935b3d","Type":"ContainerStarted","Data":"03faec73fa1a03bbbb1c32e5dc95af0cdc84b3d3e061bdcea9e141f0e113e860"}
Oct 30 00:41:27.039551 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:27.039065    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2g945" event={"ID":"06eb66bd-1808-4c71-9a97-d3da60d23201","Type":"ContainerStarted","Data":"77239363f6444fc89118a185f34bc12265277ce75e81b92999a855562f7ca41d"}
Oct 30 00:41:27.040905 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:27.040880    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7pfth" event={"ID":"f7088947-2eec-480b-9df3-285c4e6b2b59","Type":"ContainerStarted","Data":"749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867"}
Oct 30 00:41:27.082613 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:27.082544    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-blf84-qzhkv" podStartSLOduration=2.217635534 podStartE2EDuration="5.082528611s" totalImagesPullingTime="2.864893077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:27.081344419 +0000 UTC m=+35.913783258" watchObservedRunningTime="2026-10-30 00:41:27.082528611 +0000 UTC m=+35.914967439"
Oct 30 00:41:27.789312 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:27.789242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:28.046647 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:28.046567    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79" event={"ID":"87db4abb-36f5-4135-ae3a-f03cb9e2d786","Type":"ContainerStarted","Data":"ca4d6fb070b69c4aea20b46864ac47b6e88624ca3b1b30dbdf403d229a71847e"}
Oct 30 00:41:28.088978 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:28.088914    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2g945" podStartSLOduration=2.7195309930000002 podStartE2EDuration="6.088896772s" totalImagesPullingTime="3.369365779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:27.125640993 +0000 UTC m=+35.958079821" watchObservedRunningTime="2026-10-30 00:41:28.088896772 +0000 UTC m=+36.921335598"
Oct 30 00:41:28.651584 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:28.647526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:28.758332 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:28.758280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:28.896319 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:28.896266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:29.007622 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:29.007527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:29.123318 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:29.123284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:30.005518 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.005488    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pc8xn_f29d1b61-cd81-4277-af77-a44af9cc5237/dns-node-resolver/0.log"
Oct 30 00:41:30.115891 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.115836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:30.272058 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.271479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:30.274574 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.274538    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:30.297219 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.297192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:30.330074 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.330045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:30.538093 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.538003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:30.605687 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.605660    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5q95w_b995c294-397e-41e0-8a66-6e9b225ffd34/node-ca/0.log"
Oct 30 00:41:30.652464 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.652406    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79" podStartSLOduration=4.36945648 podStartE2EDuration="7.652389399s" totalImagesPullingTime="3.282932919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:28.089799097 +0000 UTC m=+36.922237925" watchObservedRunningTime="2026-10-30 00:41:30.652389399 +0000 UTC m=+39.484828204"
Oct 30 00:41:30.652630 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.652536    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hmkqx"]
Oct 30 00:41:30.658468 ip-10-0-141-17 kubenswrapper[2580]: W1030 00:41:30.658425    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac84b9b8_dee8_4ecf_acd0_6b093753b60c.slice/crio-893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311 WatchSource:0}: Error finding container 893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311: Status 404 returned error can't find the container with id 893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311
Oct 30 00:41:30.674253 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.674225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:30.775996 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.775960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:30.776182 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.776028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:30.776260 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:30.776175    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:30.776339 ip-10-0-141-17 kubenswrapper[2580]: E1030 00:41:30.776326    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:38.77630597 +0000 UTC m=+47.608744792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:30.776765 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.776738    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:30.883975 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:30.883889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:31.005231 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.005201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:31.056831 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.056606    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtsgh" event={"ID":"1adb0772-ba0b-4eb7-9322-841e3ac21ebe","Type":"ContainerStarted","Data":"39f877e5a17d1812457714a10a7a85cb6f71a136f494bd908b8e3268d238a77d"}
Oct 30 00:41:31.056831 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.056714    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:31.057953 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.057923    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hmkqx" event={"ID":"ac84b9b8-dee8-4ecf-acd0-6b093753b60c","Type":"ContainerStarted","Data":"893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311"}
Oct 30 00:41:31.102412 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.102354    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qtsgh" podStartSLOduration=34.983781188 podStartE2EDuration="39.102335722s" totalImagesPullingTime="4.118554534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:31.101999486 +0000 UTC m=+39.934438313" watchObservedRunningTime="2026-10-30 00:41:31.102335722 +0000 UTC m=+39.934774550"
Oct 30 00:41:31.117892 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.117836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:31.409450 ip-10-0-141-17 kubenswrapper[2580]: I1030 00:41:31.409423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2g945_06eb66bd-1808-4c71-9a97-d3da60d23201/serve-healthcheck-canary/0.log"
Oct 30 00:41:31.889534 ip-10-0-141-17 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 00:41:31.909392 ip-10-0-141-17 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 00:41:31.909590 ip-10-0-141-17 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 00:41:31.909648 ip-10-0-141-17 systemd[1]: kubelet.service: Consumed 2.600s CPU time.
Oct 30 00:41:31.917586 ip-10-0-141-17 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 00:41:32.007113 ip-10-0-141-17 kubenswrapper[4577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:41:32.007113 ip-10-0-141-17 kubenswrapper[4577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 00:41:32.007113 ip-10-0-141-17 kubenswrapper[4577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:41:32.007113 ip-10-0-141-17 kubenswrapper[4577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:41:32.009967 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009862    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.009967 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009963    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.009967 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009967    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.009967 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009971    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.009967 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009975    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009980    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009987    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.009994    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010002    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010007    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010012    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010017    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010021    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010026    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010030    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010034    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010038    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010042    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010046    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010050    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010053    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010058    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010062    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.010216 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010066    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010079    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010085    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010090    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010094    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010098    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010102    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010108    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010111    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010115    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010119    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010122    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010124    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010127    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010130    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010133    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010135    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010138    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010142    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010146    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.010867 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010150    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010153    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010155    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010158    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010161    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010165    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010170    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010175    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010179    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010183    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010187    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010190    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010195    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010199    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010204    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010209    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010212    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010214    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010217    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.011499 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010220    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010225    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010241    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010245    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010249    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010254    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010260    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010264    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010269    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010273    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010278    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010281    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010284    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010286    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010289    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010295    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010300    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010304    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010308    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010312    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.012075 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010316    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010320    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010324    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010328    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010332    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010336    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010340    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010345    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010350    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010353    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010357    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010361    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010365    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010370    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010374    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010377    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010379    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010382    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010384    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.012719 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010387    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010389    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010392    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010395    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.010400    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011423    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011434    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011439    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011445    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011450    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011455    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011459    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011462    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011464    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011467    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011470    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011472    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011475    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011477    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011480    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.013335 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011482    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011485    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011487    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011491    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011493    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011496    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011499    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011501    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011504    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011506    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011509    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011511    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011514    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011517    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011520    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011528    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011534    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011539    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011542    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011546    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.014024 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011549    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011553    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011557    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011561    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011583    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011587    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011601    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011607    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011611    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011616    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011620    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011622    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011625    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011628    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011630    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011633    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011636    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011638    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011640    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011643    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.014633 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011645    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011648    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011650    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011652    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011655    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011657    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011661    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011665    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011669    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011671    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011676    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011685    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011690    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011694    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011699    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011703    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011708    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011712    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011716    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.015215 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011720    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011723    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011726    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011728    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011731    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011734    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011736    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011738    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011741    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011744    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011746    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011748    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011751    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011755    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011757    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011761    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011766    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011769    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011774    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.015788 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011778    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011784    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011789    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011793    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011798    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011804    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011808    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011811    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011813    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011816    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011818    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011820    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.011823    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012516    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012526    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012531    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012536    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012540    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012544    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.016383 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012548    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012553    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012557    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012564    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012568    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012573    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012577    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012581    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012585    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012589    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012593    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012597    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012601    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012605    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012611    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012615    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012617    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012620    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012626    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.016980 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012631    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012635    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012639    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012644    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012648    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012652    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012658    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012664    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012669    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012673    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012677    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012680    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012683    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012685    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012688    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012690    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012693    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012695    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012698    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012700    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.017778 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012703    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012705    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012707    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012709    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012712    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012715    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012717    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012721    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012725    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012728    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012732    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012734    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012738    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012741    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012745    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012749    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012753    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012757    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012761    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012765    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.018423 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012768    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012772    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012776    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012779    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012783    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012787    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012791    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012794    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012799    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012802    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012805    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012808    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012811    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012813    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012816    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012820    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012824    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012828    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012832    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.019000 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012837    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012857    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012861    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012865    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012869    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012873    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012877    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012881    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012885    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012889    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012893    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012895    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012898    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012900    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012903    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012905    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012907    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012910    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012914    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012918    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.019535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012925    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.012930    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013071    4577 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013084    4577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013093    4577 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013101    4577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013108    4577 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013114    4577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013121    4577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013126    4577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013130    4577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013133    4577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013136    4577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013140    4577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013143    4577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013146    4577 flags.go:64] FLAG: --cgroup-root=""
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013148    4577 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013151    4577 flags.go:64] FLAG: --client-ca-file=""
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013154    4577 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013158    4577 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013163    4577 flags.go:64] FLAG: --cluster-domain=""
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013166    4577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013169    4577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013172    4577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 00:41:32.020314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013178    4577 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013184    4577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013189    4577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013193    4577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013199    4577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013203    4577 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013208    4577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013213    4577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013218    4577 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013222    4577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013229    4577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013234    4577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013246    4577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013251    4577 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013256    4577 flags.go:64] FLAG: --enable-server="true"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013260    4577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013266    4577 flags.go:64] FLAG: --event-burst="100"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013271    4577 flags.go:64] FLAG: --event-qps="50"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013275    4577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013280    4577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013285    4577 flags.go:64] FLAG: --eviction-hard=""
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013309    4577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013314    4577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013318    4577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013323    4577 flags.go:64] FLAG: --eviction-soft=""
Oct 30 00:41:32.021262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013327    4577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013331    4577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013336    4577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013342    4577 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013347    4577 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013351    4577 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013357    4577 flags.go:64] FLAG: --feature-gates=""
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013368    4577 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013373    4577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013379    4577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013385    4577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013391    4577 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013396    4577 flags.go:64] FLAG: --help="false"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013400    4577 flags.go:64] FLAG: --hostname-override="ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013405    4577 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013410    4577 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013416    4577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013422    4577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013428    4577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013434    4577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013437    4577 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013440    4577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013443    4577 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 00:41:32.022287 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013446    4577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013449    4577 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013453    4577 flags.go:64] FLAG: --kube-reserved=""
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013458    4577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013463    4577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013467    4577 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013472    4577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013475    4577 flags.go:64] FLAG: --lock-file=""
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013478    4577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013481    4577 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013484    4577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013492    4577 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013495    4577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013499    4577 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013502    4577 flags.go:64] FLAG: --logging-format="text"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013505    4577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013511    4577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013514    4577 flags.go:64] FLAG: --manifest-url=""
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013516    4577 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013521    4577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013527    4577 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013534    4577 flags.go:64] FLAG: --max-pods="110"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013539    4577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013542    4577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013545    4577 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 00:41:32.023080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013548    4577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013551    4577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013554    4577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013559    4577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013570    4577 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013574    4577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013579    4577 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013584    4577 flags.go:64] FLAG: --pod-cidr=""
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013588    4577 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013592    4577 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013596    4577 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013601    4577 flags.go:64] FLAG: --port="10250"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013606    4577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013609    4577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e9f9c43c57ad8f2"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013612    4577 flags.go:64] FLAG: --qos-reserved=""
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013615    4577 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013618    4577 flags.go:64] FLAG: --register-node="true"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013621    4577 flags.go:64] FLAG: --register-with-taints=""
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013629    4577 flags.go:64] FLAG: --registry-burst="10"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013634    4577 flags.go:64] FLAG: --registry-qps="5"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013639    4577 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013646    4577 flags.go:64] FLAG: --reserved-memory=""
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013653    4577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013660    4577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013665    4577 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013669    4577 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 00:41:32.023961 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013672    4577 flags.go:64] FLAG: --runonce="false"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013675    4577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013678    4577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013681    4577 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013684    4577 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013687    4577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013690    4577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013695    4577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013700    4577 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013705    4577 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013709    4577 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013713    4577 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013717    4577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013722    4577 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013727    4577 flags.go:64] FLAG: --system-cgroups=""
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013731    4577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013740    4577 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013743    4577 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013745    4577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013751    4577 flags.go:64] FLAG: --tls-min-version=""
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013754    4577 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013757    4577 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013760    4577 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013763    4577 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013767    4577 flags.go:64] FLAG: --v="2"
Oct 30 00:41:32.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013771    4577 flags.go:64] FLAG: --version="false"
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013776    4577 flags.go:64] FLAG: --vmodule=""
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013780    4577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.013783    4577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.013989    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.013995    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.013998    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014003    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014007    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014012    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014015    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014019    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014023    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014027    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014031    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014036    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014040    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014043    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014048    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014052    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.025689 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014056    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014059    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014062    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014065    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014067    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014070    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014073    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014075    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014078    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014081    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014083    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014088    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014090    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014093    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014096    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014098    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014102    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014105    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014108    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014110    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.026446 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014113    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014115    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014118    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014121    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014123    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014126    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014128    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014131    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014133    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014136    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014138    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014141    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014144    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014148    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014152    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014157    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014161    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014166    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014170    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014175    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.027244 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014179    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014183    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014188    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014194    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014196    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014199    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014202    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014205    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014211    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014215    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014219    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014223    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014227    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014230    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014234    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014238    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014242    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014246    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014249    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.027910 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014252    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014255    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014257    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014260    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014262    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014265    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014267    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014270    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014272    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014275    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014277    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014283    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014288    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014293    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014297    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014301    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014308    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014313    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014317    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.028513 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014322    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014327    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014331    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014334    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014337    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014340    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014343    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014345    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014348    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014350    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014353    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014355    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.014387    4577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014448    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014454    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.029111 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014457    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014462    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014467    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014471    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014476    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014481    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014489    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014495    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014499    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014501    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014504    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014507    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014509    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014512    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014517    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014521    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014525    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014529    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014533    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.029644 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014537    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014542    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014545    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014550    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014554    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014557    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014559    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014562    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014564    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014566    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014569    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014571    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014574    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014577    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014581    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014585    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014589    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014591    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014594    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.030261 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014596    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014598    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014601    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014604    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014606    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014608    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014611    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014614    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014619    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014621    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014624    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014626    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014629    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014632    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014635    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014638    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014641    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014644    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014647    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014650    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.030919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014652    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014655    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014657    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014660    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014662    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014664    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014667    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014669    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014672    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014675    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014677    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014680    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014682    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014684    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014687    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014690    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014692    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014694    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014697    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014699    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.031439 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014702    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014706    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014710    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014714    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014718    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014722    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014729    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014735    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014740    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014744    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014747    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014750    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014753    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014757    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014761    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014765    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014768    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014770    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014773    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.032023 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014776    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014778    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014780    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014783    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014785    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014789    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.014791    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.019350    4577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.019371    4577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.019390    4577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.019395    4577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019452    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019457    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019461    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019464    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019466    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019469    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019472    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019475    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.032615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019478    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019481    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019483    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019486    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019489    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019491    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019494    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019496    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019499    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019502    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019504    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019506    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019509    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019513    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019516    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019520    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019526    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019532    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019536    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.033221 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019541    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019545    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019551    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019555    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019560    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019564    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019569    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019572    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019575    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019578    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019580    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019583    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019585    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019588    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019590    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019592    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019606    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019609    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019612    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019625    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.033830 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019630    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019635    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019640    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019644    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019648    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019652    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019656    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019660    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019665    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019667    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019669    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019672    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019674    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019677    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019679    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019720    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019769    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019865    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019891    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019898    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.034550 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019904    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019909    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019917    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019924    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019932    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019941    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019951    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019958    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019964    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019975    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019981    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019986    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019991    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.019996    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020001    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020006    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020011    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020017    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020022    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.035219 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020027    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020034    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020038    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020050    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020055    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020062    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020067    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020074    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020078    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020083    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020090    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020096    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020102    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020107    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020110    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020114    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020122    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020126    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020131    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.035809 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020137    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.020201    4577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020679    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020688    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020693    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020698    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020703    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020707    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020716    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020721    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020726    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020730    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020734    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020738    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020743    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020748    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.036455 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020752    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020755    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020759    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020763    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020767    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020776    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020780    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020783    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020787    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020792    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020797    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020801    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020805    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020809    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020813    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020817    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020821    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020824    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020833    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.036949 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020837    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020860    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020865    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020871    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020877    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020881    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020885    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020890    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020893    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020897    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020903    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020914    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020919    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020923    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020928    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020932    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020936    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020940    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020944    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.037521 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020948    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.020965    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021024    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021033    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021039    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021043    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021050    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021055    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021060    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021065    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021268    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021296    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021301    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021304    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021307    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021309    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021312    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021315    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021318    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021321    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.038173 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021323    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021327    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021329    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021332    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021334    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021337    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021340    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021342    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021345    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021347    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021350    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021353    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021356    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021358    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021361    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021364    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021367    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021369    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021372    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021375    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.038920 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021377    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021380    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021383    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021386    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021389    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021391    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021394    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021397    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021402    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021406    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021410    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021414    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021418    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021423    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.021479    4577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021540    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:41:32.039464 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021548    4577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021554    4577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021558    4577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021563    4577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021568    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021573    4577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021578    4577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021582    4577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021587    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021589    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021592    4577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021594    4577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021598    4577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021601    4577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021604    4577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021607    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021609    4577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021612    4577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021615    4577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:41:32.039925 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021622    4577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021627    4577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021631    4577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021636    4577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021641    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021646    4577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021649    4577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021652    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021655    4577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021657    4577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021660    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021663    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021665    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021667    4577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021671    4577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021675    4577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021682    4577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021686    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021688    4577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021691    4577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:41:32.040523 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021693    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021696    4577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021699    4577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021701    4577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021704    4577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021707    4577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021710    4577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021712    4577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021715    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021717    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021720    4577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021723    4577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021727    4577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021729    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021732    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021734    4577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021737    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021739    4577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021741    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:41:32.041120 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021744    4577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021746    4577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021751    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021753    4577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021756    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021758    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021761    4577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021763    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021766    4577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021769    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021772    4577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021774    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021777    4577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021779    4577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021783    4577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021787    4577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021790    4577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021793    4577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021796    4577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:41:32.041678 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021798    4577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021801    4577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021805    4577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021808    4577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021811    4577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021814    4577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021821    4577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021825    4577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021830    4577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021834    4577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021838    4577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021857    4577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021861    4577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021866    4577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021870    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021875    4577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021879    4577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021886    4577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021892    4577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:41:32.042374 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021896    4577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021900    4577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021904    4577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021910    4577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021914    4577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021918    4577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021921    4577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021924    4577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:32.021926    4577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.021995    4577 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.023190    4577 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.023261    4577 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.023487    4577 server.go:1014] "Starting client certificate rotation"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.023635    4577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.023674    4577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 00:35:52 +0000 UTC" deadline="2028-05-29 14:39:46.461612092 +0000 UTC"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.023710    4577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13861h58m14.437904324s"
Oct 30 00:41:32.043156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.024133    4577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:41:32.043818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.025607    4577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:41:32.043818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.027353    4577 log.go:25] "Validated CRI v1 runtime API"
Oct 30 00:41:32.043818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.029041    4577 log.go:25] "Validated CRI v1 image API"
Oct 30 00:41:32.043818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.029928    4577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 00:41:32.043818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.031681    4577 fs.go:135] Filesystem UUIDs: map[3dc1538c-df45-4d3b-97b9-8b2a072572b2:/dev/nvme0n1p3 62064635-10b0-41da-984b-0e40437c29ae:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 00:41:32.044074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.031705    4577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/43c6311c7124ee6e289fe606e7eb3c45a47b8a5f1b4a68afb2f7f454bd49926e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/43c6311c7124ee6e289fe606e7eb3c45a47b8a5f1b4a68afb2f7f454bd49926e/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311/userdata/shm major:0 minor:264 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf/volumes/kubernetes.io~projected/kube-api-access-klrps:{mountpoint:/var/lib/kubelet/pods/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf/volumes/kubernetes.io~projected/kube-api-access-klrps major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06eb66bd-1808-4c71-9a97-d3da60d23201/volumes/kubernetes.io~projected/kube-api-access-xf5ws:{mountpoint:/var/lib/kubelet/pods/06eb66bd-1808-4c71-9a97-d3da60d23201/volumes/kubernetes.io~projected/kube-api-access-xf5ws major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06eb66bd-1808-4c71-9a97-d3da60d23201/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/06eb66bd-1808-4c71-9a97-d3da60d23201/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ad924ba-4fd0-4a90-b699-01c066c8de59/volumes/kubernetes.io~projected/kube-api-access-vvl8d:{mountpoint:/var/lib/kubelet/pods/0ad924ba-4fd0-4a90-b699-01c066c8de59/volumes/kubernetes.io~projected/kube-api-access-vvl8d major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1adb0772-ba0b-4eb7-9322-841e3ac21ebe/volumes/kubernetes.io~projected/kube-api-access-pgmpw:{mountpoint:/var/lib/kubelet/pods/1adb0772-ba0b-4eb7-9322-841e3ac21ebe/volumes/kubernetes.io~projected/kube-api-access-pgmpw major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2e79761e-c9f1-494b-ba28-941db904b171/volumes/kubernetes.io~projected/kube-api-access-87cqm:{mountpoint:/var/lib/kubelet/pods/2e79761e-c9f1-494b-ba28-941db904b171/volumes/kubernetes.io~projected/kube-api-access-87cqm major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/490ed73e-84fe-4053-bec4-43f479935b3d/volumes/kubernetes.io~projected/kube-api-access-x7czs:{mountpoint:/var/lib/kubelet/pods/490ed73e-84fe-4053-bec4-43f479935b3d/volumes/kubernetes.io~projected/kube-api-access-x7czs major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7498e2f5-ea7b-4238-9111-bde1648f2fd1/volumes/kubernetes.io~projected/kube-api-access-7zp9v:{mountpoint:/var/lib/kubelet/pods/7498e2f5-ea7b-4238-9111-bde1648f2fd1/volumes/kubernetes.io~projected/kube-api-access-7zp9v major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7be17326-1cdc-43a1-b333-297b136611bf/volumes/kubernetes.io~projected/kube-api-access-f4bqb:{mountpoint:/var/lib/kubelet/pods/7be17326-1cdc-43a1-b333-297b136611bf/volumes/kubernetes.io~projected/kube-api-access-f4bqb major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~projected/kube-api-access-m969p:{mountpoint:/var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~projected/kube-api-access-m969p major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87db4abb-36f5-4135-ae3a-f03cb9e2d786/volumes/kubernetes.io~projected/kube-api-access-sg9gh:{mountpoint:/var/lib/kubelet/pods/87db4abb-36f5-4135-ae3a-f03cb9e2d786/volumes/kubernetes.io~projected/kube-api-access-sg9gh major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87db4abb-36f5-4135-ae3a-f03cb9e2d786/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/87db4abb-36f5-4135-ae3a-f03cb9e2d786/volumes/kubernetes.io~secret/signing-key major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac84b9b8-dee8-4ecf-acd0-6b093753b60c/volumes/kubernetes.io~projected/kube-api-access-tzc66:{mountpoint:/var/lib/kubelet/pods/ac84b9b8-dee8-4ecf-acd0-6b093753b60c/volumes/kubernetes.io~projected/kube-api-access-tzc66 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac84b9b8-dee8-4ecf-acd0-6b093753b60c/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/ac84b9b8-dee8-4ecf-acd0-6b093753b60c/volumes/kubernetes.io~secret/metrics-tls major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b995c294-397e-41e0-8a66-6e9b225ffd34/volumes/kubernetes.io~projected/kube-api-access-2927s:{mountpoint:/var/lib/kubelet/pods/b995c294-397e-41e0-8a66-6e9b225ffd34/volumes/kubernetes.io~projected/kube-api-access-2927s major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c9c1e69b-d783-4731-8eba-61f21232cb31/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/c9c1e69b-d783-4731-8eba-61f21232cb31/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d2907a2d-9dbd-4dae-93d6-841dfcd2a596/volumes/kubernetes.io~projected/kube-api-access-29hz7:{mountpoint:/var/lib/kubelet/pods/d2907a2d-9dbd-4dae-93d6-841dfcd2a596/volumes/kubernetes.io~projected/kube-api-access-29hz7 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f29d1b61-cd81-4277-af77-a44af9cc5237/volumes/kubernetes.io~projected/kube-api-access-dhjwm:{mountpoint:/var/lib/kubelet/pods/f29d1b61-cd81-4277-af77-a44af9cc5237/volumes/kubernetes.io~projected/kube-api-access-dhjwm major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f50690df-0bf7-473a-a1e9-df93dacf3afa/volumes/kubernetes.io~projected/kube-api-access-67jsn:{mountpoint:/var/lib/kubelet/pods/f50690df-0bf7-473a-a1e9-df93dacf3afa/volumes/kubernetes.io~projected/kube-api-access-67jsn major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7088947-2eec-480b-9df3-285c4e6b2b59/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/f7088947-2eec-480b-9df3-285c4e6b2b59/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volumes/kubernetes.io~projected/kube-api-access-ggq7r:{mountpoint:/var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volumes/kubernetes.io~projected/kube-api-access-ggq7r major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/6d4a98e91367336a0aff9f7c98c90c3cde70eeff52fff20e2e24768381dc949f/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/84196ea46e24ff4eb8235fdeaa7478fa3e5c258adb2e0a01fbf436f3c48e2941/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/c6113cdfc0aeac50000153907c8f80556989ee55071d7050a7cd48341689f4bf/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/4ee9e3c2ba845fba229158f9e0c6760930c7cae8d9a79f7ca02400b31a85965a/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/8143e27b5cbeec9e7ff089d1bdf4e1b17c22ed35ade8c2f79c06d4689a3859a6/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/790a17053f9efa6fc34da801194a2385a57d39c5fb7cd632b319743c9b0b7e41/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/55ecececb4bd5bcd2065f28eef7ed6cc5cf1c747d8c3a77e79c90a3dddaa6b0d/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/ac21296b65e2d37b085349f82be7dea883a38f1bda2b506598d895f08272f928/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/dada317e7a48dec9173ed4c0db04caf6c6364172796a80683280c5dff3905013/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/602c6d2bc3b024ed8297fc5677880762995b194b79953f6544a2c8b8ff3dbe2c/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/6681c281f4a452ccefd0aa5d267ac4e84f0733b8ce3e5531b8fa8423f7290ea3/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/f99cb714ddb85412031829ccf2b21501c295514122a7d12abb0a7eeb5f461111/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/ccfc1928abf972de6864ed6235a5c12314380d763b67981d6326e2f2b713fb43/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/92ffe9ca39feeb1a46dd1bbf933c9b02e0962751f4e80c8d6ffe3efe1a0c8094/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-159:{mountpoint:/var/lib/containers/storage/overlay/6fdb1c33c73f762f88014152aee1bebc6803b7fca5c4a081d176b4ca50815b00/merged major:0 minor:159 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/73db6f575bae6a7e272a0c41d6abb039dfe0fc6e948ad450696ffeb0f568c7c8/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/6ab1b9aebb8dd0a6becde025c212d6d3c07339b725019181379d7e7fd4b77ab3/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/8f588655c71ca847eb7bf5ca887260e64cfad51effcff4fc515dbba2455176fa/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/892e60484f39a6677b7c3e5e9df4bb7f856a56713e51d101628bd2830ddac9da/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c1ef8893b663a51034f16094291dcc406a6f8e36e3dfc1cea2e84ba92ac2fb30/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/0ad59ec5cc9a124b8cb5330733ea95c1f2e4149ace509afd49220a44cab8659e/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/3ab63a7ea6df46fda80785547ca403522599aa84fbfcbf076ec24075dccce661/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/b1e688c592060cbedeb1362b14885216bff7d848f8faba9186083c740376aee0/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/a590fd42306c1f51087710c12c76e7e428e021693f0e742260f70843ee7fa7b7/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/d96d86996f09ebd421511e1d8d529da701fc53836bdaecddad6d529d354aa3f0/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/84a1de5b5f879522557cdb9561e0c6327abd520a1609533a121e4467c3e42d93/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/6ae35156a1a3c5b06db591ccc35e9d4ac488ce2e7976107f579750826d1c5344/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/72e53bd5898b3dfc4c12834d1d9a80ecc39c27702210587b0173cc00e8208044/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/75665a982d80e82dcc656856e66a8c3efad1d7db25214cc8a7cf1ddde48a24d2/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/4bc2746320093fd56ae89acb980a9da65933c7c24fc3a66fa016e1e42aebd65e/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/ff940535cd00fcd6905a0b739c4fc6b5990b1a8a009db3202a0f097490068848/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/1fa330b40fe1bbf79477e66bb1c06ccf4884ca60f12e2e29d0d1068c8be3adaa/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/dd7e70ad4c0792b2d10190e6aa93fb149bdaf78807847386d1ac14e3d240fac0/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/2b024d42782cb7472a910dfa09480a1f096c515a7088e4ddb0931a03887af115/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/d67df026f159e92568c59d0e361dc110a7a74e49a265d525e2908749aef2c300/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/396f00aaf25b43702ac535f340806d8f6235c12df4b3588ebfad799b26edd384/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/b5a22ce4fcb651b28845f39f1c2cdafe6a02d23d59626851a4b3e1b9b652b611/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/4d0d7c018b61f44401741875dfc6e5bf7cf58672d7d93a8798b3eb0f9116d429/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-77:{mountpoint:/var/lib/containers/storage/overlay/adf29381daf7c28a2c2a686ad9fd142598576939c021c5eca82fb4f3b63576c3/merged major:0 minor:77 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/981142fc9ca554fb51b629fdb5b68062d8480d71ca6a1c0c8f3d75c551997982/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/d429f41f5e743cf2f3d03f5cc6856666901095a38a3d206767dc9a89263c1518/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/87fd8ab77f41af00634626d2e881c0886ddf5b35e58dec6294bda5b955eb8982/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/c94c44502352517e7087182fb48626d9270d2bf1be2196aa70d6c539db5f0650/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/4b6da531da5acc11930e00e48ad0756c2056c767c490a1b96505ccd852446806/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.039528    4577 manager.go:217] Machine: {Timestamp:2026-10-30 00:41:32.038959854 +0000 UTC m=+0.068270076 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196856 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2224c25fe8109cdd7355f2bd2d3b9a SystemUUID:ec2224c2-5fe8-109c-dd73-55f2bd2d3b9a BootID:ed687dff-44da-4620-96f7-dd80fd38efd6 Filesystems:[{Device:/run/containers/storage/overlay-containers/43c6311c7124ee6e289fe606e7eb3c45a47b8a5f1b4a68afb2f7f454bd49926e/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d2907a2d-9dbd-4dae-93d6-841dfcd2a596/volumes/kubernetes.io~projected/kube-api-access-29hz7 DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-77 DeviceMajor:0 DeviceMinor:77 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~projected/kube-api-access-m969p DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b995c294-397e-41e0-8a66-6e9b225ffd34/volumes/kubernetes.io~projected/kube-api-access-2927s DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/490ed73e-84fe-4053-bec4-43f479935b3d/volumes/kubernetes.io~projected/kube-api-access-x7czs DeviceMajor:0 DeviceMinor:207 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f7088947-2eec-480b-9df3-285c4e6b2b59/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:225 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06eb66bd-1808-4c71-9a97-d3da60d23201/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f29d1b61-cd81-4277-af77-a44af9cc5237/volumes/kubernetes.io~projected/kube-api-access-dhjwm DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1adb0772-ba0b-4eb7-9322-841e3ac21ebe/volumes/kubernetes.io~projected/kube-api-access-pgmpw DeviceMajor:0 DeviceMinor:230 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/73122a3aac1f8901908261747056b5f62ad28a0094ac99ad4d3472e9fed50bf3/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-159 DeviceMajor:0 DeviceMinor:159 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06eb66bd-1808-4c71-9a97-d3da60d23201/volumes/kubernetes.io~projected/kube-api-access-xf5ws DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7498e2f5-ea7b-4238-9111-bde1648f2fd1/volumes/kubernetes.io~projected/kube-api-access-7zp9v DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7be17326-1cdc-43a1-b333-297b136611bf/volumes/kubernetes.io~projected/kube-api-access-f4bqb DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0d46ad211aa7b25481d69fb25ac17d41b92fa17976138bbd5f3cf74fa169e04a/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/c9c1e69b-d783-4731-8eba-61f21232cb31/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ac84b9b8-dee8-4ecf-acd0-6b093753b60c/volumes/kubernetes.io~projected/kube-api-access-tzc66 DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0ad924ba-4fd0-4a90-b699-01c066c8de59/volumes/kubernetes.io~projected/kube-api-access-vvl8d DeviceMajor:0 DeviceMinor:206 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf/volumes/kubernetes.io~projected/kube-api-access-klrps DeviceMajor:0 DeviceMinor:212 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/87db4abb-36f5-4135-ae3a-f03cb9e2d786/volumes/kubernetes.io~projected/kube-api-access-sg9gh DeviceMajor:0 DeviceMinor:218 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/639cbfbd465fa2d83b646bd8b6506d3a618315700ed736017beb0af4bac843d1/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81a8cc41308ffa7857806fef81ab8f05a67d0917cc48e46990ce3087e38f558a/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/81f12500-d8ab-44de-bb4f-a5a987bddb69/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5d6e44bd90a52cdb43b5919aedd1670e094652d50d0f67b85817751f7e4955e6/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/893cd5faa1c03b457e67048833315fbaf0b14c8f33bb50a4190812648082d311/userdata/shm DeviceMajor:0 DeviceMinor:264 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f50690df-0bf7-473a-a1e9-df93dacf3afa/volumes/kubernetes.io~projected/kube-api-access-67jsn DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f932ce1bf4048b0f044398fd4791cddd24bec060eb85acdf87388339e7ce0bc6/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volumes/kubernetes.io~projected/kube-api-access-ggq7r DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1133c81a30267669b392a0c42cb3d6dab47b92a3accc711366e8f6fd0f5bbdf9/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac84b9b8-dee8-4ecf-acd0-6b093753b60c/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:259 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2e79761e-c9f1-494b-ba28-941db904b171/volumes/kubernetes.io~projected/kube-api-access-87cqm DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8226f04e4657442673963003d1f282a0e6dfca5167a0e831e68060637633527e/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1adadc89cabf57761a218a6a196cb1bb1f8b38bc212f1db16ec0a2e7f5df9c14/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/73b365e57d8ecf80f5f4142b1849712b4694250e0f408a2c3a55ddd3574bf08e/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb453dade4ceab52e83daf0d5ee755dd038457e02e5d4f44535b55c884e33caf/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/fbfacf74-0913-413c-b764-ec9afff26246/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/749235eff19eef2b01a6ed6f278a2aec06f50e40695cc9d1dbe63bbe13957867/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/87db4abb-36f5-4135-ae3a-f03cb9e2d786/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:217 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1133c81a3026766 MacAddress:e6:83:23:6b:c7:65 Speed:10000 Mtu:8901} {Name:56a716aa989186a MacAddress:f2:33:23:87:02:90 Speed:10000 Mtu:8901} {Name:73122a3aac1f890 MacAddress:02:8d:03:11:00:fa Speed:10000 Mtu:8901} {Name:749235eff19eef2 MacAddress:e2:ee:d5:18:58:ff Speed:10000 Mtu:8901} {Name:81a8cc41308ffa7 MacAddress:f2:70:ce:21:ca:8f Speed:10000 Mtu:8901} {Name:893cd5faa1c03b4 MacAddress:42:6c:44:b4:e7:c7 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d3:9a:bc:e7 Speed:0 Mtu:9001} {Name:br-int MacAddress:de:bb:6a:ea:96:8f Speed:0 Mtu:8901} {Name:ens5 MacAddress:02:ff:d3:9a:bc:e7 Speed:0 Mtu:9001} {Name:fb453dade4ceab5 MacAddress:fe:c5:2c:48:05:82 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:72:7c:56:5d:38:b4 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:8a:8a:5b:85:78:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.039878    4577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.039941    4577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040224    4577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040245    4577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040396    4577 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040404    4577 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040418    4577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040434    4577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040524    4577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040706    4577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040823    4577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040879    4577 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040894    4577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040905    4577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040912    4577 kubelet.go:405] "Adding apiserver pod source"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.040920    4577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.042974    4577 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 00:41:32.044305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.042994    4577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:41:32.045018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.044734    4577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 00:41:32.056262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056244    4577 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 00:41:32.056262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056258    4577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:41:32.056437 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056366    4577 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 00:41:32.056575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056562    4577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 00:41:32.056628 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056592    4577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 00:41:32.056725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056704    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 00:41:32.056725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056724    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056732    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056737    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056744    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056750    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056755    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056761    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056768    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056774    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056783    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 00:41:32.056817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056793    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 00:41:32.057302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056826    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 00:41:32.057302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.056834    4577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 00:41:32.060618 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.060602    4577 server.go:1263] "Started kubelet"
Oct 30 00:41:32.061121 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.061067    4577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 00:41:32.061218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.061121    4577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 00:41:32.061133 ip-10-0-141-17 systemd[1]: Started Kubernetes Kubelet.
Oct 30 00:41:32.063066 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.062951    4577 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 00:41:32.063542 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.063525    4577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 00:41:32.064226 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.064208    4577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 00:41:32.065372 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.065353    4577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 00:41:32.066586 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:32.066558    4577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-17.ec2.internal\" not found"
Oct 30 00:41:32.066669 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.066604    4577 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 00:41:32.066669 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.066612    4577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 00:41:32.066865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.066832    4577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 00:41:32.068514 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068474    4577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 00:35:54 +0000 UTC" deadline="2028-07-31 23:07:52.062392114 +0000 UTC"
Oct 30 00:41:32.068514 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068513    4577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15382h26m19.993883487s"
Oct 30 00:41:32.068657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068581    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e80bbb-3f65-4513-8fe3-8c43c2fa83bf" volumeName="kubernetes.io/projected/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf-kube-api-access-klrps" seLinuxMountContext=""
Oct 30 00:41:32.068657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068614    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87db4abb-36f5-4135-ae3a-f03cb9e2d786" volumeName="kubernetes.io/configmap/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-cabundle" seLinuxMountContext=""
Oct 30 00:41:32.068657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068629    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="490ed73e-84fe-4053-bec4-43f479935b3d" volumeName="kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs" seLinuxMountContext=""
Oct 30 00:41:32.068657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068643    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87db4abb-36f5-4135-ae3a-f03cb9e2d786" volumeName="kubernetes.io/projected/87db4abb-36f5-4135-ae3a-f03cb9e2d786-kube-api-access-sg9gh" seLinuxMountContext=""
Oct 30 00:41:32.068657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068655    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06eb66bd-1808-4c71-9a97-d3da60d23201" volumeName="kubernetes.io/projected/06eb66bd-1808-4c71-9a97-d3da60d23201-kube-api-access-xf5ws" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068667    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1adb0772-ba0b-4eb7-9322-841e3ac21ebe" volumeName="kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068680    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e79761e-c9f1-494b-ba28-941db904b171" volumeName="kubernetes.io/projected/2e79761e-c9f1-494b-ba28-941db904b171-kube-api-access-87cqm" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068691    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7be17326-1cdc-43a1-b333-297b136611bf" volumeName="kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-binary-copy" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068713    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81f12500-d8ab-44de-bb4f-a5a987bddb69" volumeName="kubernetes.io/projected/81f12500-d8ab-44de-bb4f-a5a987bddb69-kube-api-access-m969p" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068724    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac84b9b8-dee8-4ecf-acd0-6b093753b60c" volumeName="kubernetes.io/projected/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-kube-api-access-tzc66" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068734    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac84b9b8-dee8-4ecf-acd0-6b093753b60c" volumeName="kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068748    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b995c294-397e-41e0-8a66-6e9b225ffd34" volumeName="kubernetes.io/projected/b995c294-397e-41e0-8a66-6e9b225ffd34-kube-api-access-2927s" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068762    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ad924ba-4fd0-4a90-b699-01c066c8de59" volumeName="kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068785    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ad924ba-4fd0-4a90-b699-01c066c8de59" volumeName="kubernetes.io/projected/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-api-access-vvl8d" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068798    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7be17326-1cdc-43a1-b333-297b136611bf" volumeName="kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068809    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7be17326-1cdc-43a1-b333-297b136611bf" volumeName="kubernetes.io/projected/7be17326-1cdc-43a1-b333-297b136611bf-kube-api-access-f4bqb" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068831    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81f12500-d8ab-44de-bb4f-a5a987bddb69" volumeName="kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-tmp" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068877    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87db4abb-36f5-4135-ae3a-f03cb9e2d786" volumeName="kubernetes.io/secret/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-key" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068889    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac84b9b8-dee8-4ecf-acd0-6b093753b60c" volumeName="kubernetes.io/empty-dir/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-tmp-dir" seLinuxMountContext=""
Oct 30 00:41:32.068894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068902    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9c1e69b-d783-4731-8eba-61f21232cb31" volumeName="kubernetes.io/secret/c9c1e69b-d783-4731-8eba-61f21232cb31-agent-certs" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068914    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7498e2f5-ea7b-4238-9111-bde1648f2fd1" volumeName="kubernetes.io/projected/7498e2f5-ea7b-4238-9111-bde1648f2fd1-kube-api-access-7zp9v" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068926    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81f12500-d8ab-44de-bb4f-a5a987bddb69" volumeName="kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-tuned" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068938    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d2907a2d-9dbd-4dae-93d6-841dfcd2a596" volumeName="kubernetes.io/projected/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-kube-api-access-29hz7" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068949    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f29d1b61-cd81-4277-af77-a44af9cc5237" volumeName="kubernetes.io/projected/f29d1b61-cd81-4277-af77-a44af9cc5237-kube-api-access-dhjwm" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068959    4577 factory.go:55] Registering systemd factory
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068977    4577 factory.go:223] Registration of the systemd container factory successfully
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.068960    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7088947-2eec-480b-9df3-285c4e6b2b59" volumeName="kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069042    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbfacf74-0913-413c-b764-ec9afff26246" volumeName="kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069069    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ad924ba-4fd0-4a90-b699-01c066c8de59" volumeName="kubernetes.io/empty-dir/0ad924ba-4fd0-4a90-b699-01c066c8de59-data-volume" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069090    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f29d1b61-cd81-4277-af77-a44af9cc5237" volumeName="kubernetes.io/empty-dir/f29d1b61-cd81-4277-af77-a44af9cc5237-tmp-dir" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069112    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f50690df-0bf7-473a-a1e9-df93dacf3afa" volumeName="kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-cni-binary-copy" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069126    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f50690df-0bf7-473a-a1e9-df93dacf3afa" volumeName="kubernetes.io/projected/f50690df-0bf7-473a-a1e9-df93dacf3afa-kube-api-access-67jsn" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069140    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbfacf74-0913-413c-b764-ec9afff26246" volumeName="kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-env-overrides" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069152    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbfacf74-0913-413c-b764-ec9afff26246" volumeName="kubernetes.io/projected/fbfacf74-0913-413c-b764-ec9afff26246-kube-api-access-ggq7r" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069165    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbfacf74-0913-413c-b764-ec9afff26246" volumeName="kubernetes.io/secret/fbfacf74-0913-413c-b764-ec9afff26246-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069177    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="490ed73e-84fe-4053-bec4-43f479935b3d" volumeName="kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069188    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="490ed73e-84fe-4053-bec4-43f479935b3d" volumeName="kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069202    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="490ed73e-84fe-4053-bec4-43f479935b3d" volumeName="kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069217    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7be17326-1cdc-43a1-b333-297b136611bf" volumeName="kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069230    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b995c294-397e-41e0-8a66-6e9b225ffd34" volumeName="kubernetes.io/configmap/b995c294-397e-41e0-8a66-6e9b225ffd34-serviceca" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069253    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9c1e69b-d783-4731-8eba-61f21232cb31" volumeName="kubernetes.io/configmap/c9c1e69b-d783-4731-8eba-61f21232cb31-konnectivity-ca" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069269    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f50690df-0bf7-473a-a1e9-df93dacf3afa" volumeName="kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-daemon-config" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069301    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06eb66bd-1808-4c71-9a97-d3da60d23201" volumeName="kubernetes.io/secret/06eb66bd-1808-4c71-9a97-d3da60d23201-cert" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069313    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7498e2f5-ea7b-4238-9111-bde1648f2fd1" volumeName="kubernetes.io/configmap/7498e2f5-ea7b-4238-9111-bde1648f2fd1-iptables-alerter-script" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069329    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac84b9b8-dee8-4ecf-acd0-6b093753b60c" volumeName="kubernetes.io/configmap/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-config-volume" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069342    4577 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbfacf74-0913-413c-b764-ec9afff26246" volumeName="kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-config" seLinuxMountContext=""
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069354    4577 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069361    4577 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069378    4577 factory.go:153] Registering CRI-O factory
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069393    4577 factory.go:223] Registration of the crio container factory successfully
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069444    4577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069469    4577 factory.go:103] Registering Raw factory
Oct 30 00:41:32.069552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.069483    4577 manager.go:1199] Started watching for new ooms in manager
Oct 30 00:41:32.070637 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.070024    4577 manager.go:319] Starting recovery of all containers
Oct 30 00:41:32.071701 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.071682    4577 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 00:41:32.100962 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.100910    4577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 00:41:32.101207 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.101182    4577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:41:32.103128 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.103076    4577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 00:41:32.103211 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.103131    4577 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 00:41:32.103211 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.103154    4577 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 00:41:32.103709 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:32.103681    4577 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 00:41:32.115561 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.115540    4577 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 00:41:32.115922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.115903    4577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:41:32.129664 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.129639    4577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:41:32.130825 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.130799    4577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:41:32.136688 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.136660    4577 generic.go:356] "Generic (PLEG): container finished" podID="0dd5d59005c843cb87cd033c834ec196" containerID="bf19ad2be000c1ddfe2218b7a9c826fc982aa713d4260c223062d03837e293a6" exitCode=0
Oct 30 00:41:32.153253 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153206    4577 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="030d8eed4afdb444c8d8f7589acb229cf86d68ad1d0e65a876f73f652f78ef9f" exitCode=0
Oct 30 00:41:32.153253 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153229    4577 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="c1807d0197c1150ab6d6fb0899f8003a84781ea8ecfef1d745866098d79e3683" exitCode=0
Oct 30 00:41:32.153253 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153245    4577 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="e1f70b3a3420080b9e8724191834b21b63db3dcbce4b60cd7d463e211d67ab5f" exitCode=0
Oct 30 00:41:32.153253 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153254    4577 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="27722fae50d12a818c8678871719dedd2966401791db522efe2c94a341d1285d" exitCode=0
Oct 30 00:41:32.153522 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153265    4577 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="d792de6a3f23cfc7bf85ebcf54b8a22022593b388a78cd2ce05ca17ea2cdab4b" exitCode=0
Oct 30 00:41:32.153522 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153292    4577 generic.go:356] "Generic (PLEG): container finished" podID="7be17326-1cdc-43a1-b333-297b136611bf" containerID="4cf698abb7fd8da55955dd9d0b0eeba19c8a375c918f5224ab340daff1be76d0" exitCode=0
Oct 30 00:41:32.153873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.153835    4577 manager.go:324] Recovery completed
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176135    4577 cpu_manager.go:226] "Starting" policy="none"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176152    4577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176172    4577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176312    4577 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176326    4577 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176347    4577 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176355    4577 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176362    4577 policy_none.go:50] "Start"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176373    4577 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176385    4577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176547    4577 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176556    4577 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 00:41:32.177044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.176563    4577 policy_none.go:44] "Start"
Oct 30 00:41:32.182092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.182042    4577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 00:41:32.185125 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185107    4577 manager.go:341] "Starting Device Plugin manager"
Oct 30 00:41:32.185216 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:32.185136    4577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 00:41:32.185216 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185153    4577 server.go:87] "Starting device plugin registration server"
Oct 30 00:41:32.185508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185494    4577 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 00:41:32.185593 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185507    4577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 00:41:32.185653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185612    4577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 00:41:32.185760 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185716    4577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 00:41:32.185911 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185767    4577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 00:41:32.185911 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185772    4577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 00:41:32.186026 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.185810    4577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T00:41:32.185756454Z","UUID":"2938d9c4-9658-424f-80e3-a41496241bdf","Handler":null,"Name":"","Endpoint":""}
Oct 30 00:41:32.187190 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.187172    4577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 00:41:32.187273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.187208    4577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 00:41:32.187332 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:32.187297    4577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 00:41:32.206223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206177    4577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"]
Oct 30 00:41:32.206870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206813    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" event={"ID":"0dd5d59005c843cb87cd033c834ec196","Type":"ContainerStarted","Data":"7764e08df45ea69c98765b7e67c46e0af987f79dd8c9aff8d04f528187d08286"}
Oct 30 00:41:32.206971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" event={"ID":"0dd5d59005c843cb87cd033c834ec196","Type":"ContainerDied","Data":"bf19ad2be000c1ddfe2218b7a9c826fc982aa713d4260c223062d03837e293a6"}
Oct 30 00:41:32.206971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206899    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal" event={"ID":"0dd5d59005c843cb87cd033c834ec196","Type":"ContainerStarted","Data":"9bc830f4d0a0ee5b40e42156eeeb02ee52f252075aabb51f687eb28c8032153e"}
Oct 30 00:41:32.206971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206907    4577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.206971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206958    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal" event={"ID":"ab07c2e2299f7ce9c9ccf7ec1f988b83","Type":"ContainerStarted","Data":"b060ed398719a24a973572837c91e0c1a0fa24fe5d9bc82bc767e31ff0b508ed"}
Oct 30 00:41:32.207165 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.206974    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal" event={"ID":"ab07c2e2299f7ce9c9ccf7ec1f988b83","Type":"ContainerStarted","Data":"0b92d24eddc43bc4de4b1bfe23bd8e4c95310b2878eb53dd99f528aaa42a05a7"}
Oct 30 00:41:32.207165 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.207065    4577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.232545 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.232507    4577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:41:32.232666 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:32.232568    4577 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.233158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.233137    4577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:41:32.233264 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:32.233185    4577 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.286123 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.286051    4577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:41:32.287529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.287500    4577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:41:32.287529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.287532    4577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:41:32.287684 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.287547    4577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-17.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:41:32.287684 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.287587    4577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.316727 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.316696    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.316727 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.316727    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.316982 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.316742    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab07c2e2299f7ce9c9ccf7ec1f988b83-config\") pod \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" (UID: \"ab07c2e2299f7ce9c9ccf7ec1f988b83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.326325 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.326295    4577 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.326462 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.326375    4577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.417202 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.417173    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.417202 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.417206    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.417551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.417222    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab07c2e2299f7ce9c9ccf7ec1f988b83-config\") pod \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" (UID: \"ab07c2e2299f7ce9c9ccf7ec1f988b83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.417551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.417305    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.417551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.417329    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0dd5d59005c843cb87cd033c834ec196-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" (UID: \"0dd5d59005c843cb87cd033c834ec196\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:32.417551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:32.417360    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab07c2e2299f7ce9c9ccf7ec1f988b83-config\") pod \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" (UID: \"ab07c2e2299f7ce9c9ccf7ec1f988b83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:33.042614 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.042578    4577 apiserver.go:52] "Watching apiserver"
Oct 30 00:41:33.088381 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.088353    4577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 00:41:33.088820 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.088801    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-blf84-qzhkv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal","openshift-ingress-canary/ingress-canary-2g945","openshift-multus/multus-additional-cni-plugins-ms722","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4","openshift-network-operator/iptables-alerter-clgnx","openshift-network-diagnostics/network-check-target-qtsgh","openshift-ovn-kubernetes/ovnkube-node-xp86c","kube-system/konnectivity-agent-2ttqh","kube-system/global-pull-secret-syncer-7pfth","openshift-multus/network-metrics-daemon-s2jdz","openshift-dns/dns-default-hmkqx","openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p","openshift-multus/multus-dslp8","openshift-service-ca/service-ca-5fffc97bdf-dpp79","openshift-cluster-node-tuning-operator/tuned-57vkf","openshift-insights/insights-runtime-extractor-2v4fv","openshift-dns/node-resolver-pc8xn","openshift-image-registry/node-ca-5q95w"]
Oct 30 00:41:33.092391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.092371    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.093606 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.093591    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.109718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109684    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jgsn4\""
Oct 30 00:41:33.109718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109710    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 00:41:33.109927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109767    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 00:41:33.109927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109781    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z5p7p\""
Oct 30 00:41:33.109927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109787    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5b9kk\""
Oct 30 00:41:33.109927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109812    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 00:41:33.109927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.109811    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 00:41:33.110307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110293    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 00:41:33.110378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110333    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.110440 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110414    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.110440 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110432    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.110514 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110432    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.110514 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110440    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.110514 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110462    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.110638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110544    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z4qxk\""
Oct 30 00:41:33.110638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110582    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 00:41:33.110638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110596    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.110638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110610    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.110638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110622    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 00:41:33.110638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110634    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 00:41:33.110811 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110634    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7sn5q\""
Oct 30 00:41:33.110811 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110651    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-cg6zp\""
Oct 30 00:41:33.110811 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.110653    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.119803 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.119791    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.119954 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.119940    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 00:41:33.120147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.120135    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.121909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.121874    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 00:41:33.121909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.121898    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 00:41:33.121909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.121883    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-748hz\""
Oct 30 00:41:33.121909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.121886    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wwjm6\""
Oct 30 00:41:33.121909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.121894    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 00:41:33.121909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.121907    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-92qkq\""
Oct 30 00:41:33.122370 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.122356    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-wm8np\""
Oct 30 00:41:33.122574 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.122556    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 00:41:33.122641 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.122601    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qs9gb\""
Oct 30 00:41:33.122641 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.122635    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-f5gxs\""
Oct 30 00:41:33.123054 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.123041    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 00:41:33.123209 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.123197    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 00:41:33.123701 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.123687    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 00:41:33.123742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.123728    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.123791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.123779    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kdklf\""
Oct 30 00:41:33.124090 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.124068    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.124993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.124979    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l8pvs\""
Oct 30 00:41:33.125099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.125077    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 00:41:33.125144 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.125099    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.125144 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.125101    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 00:41:33.125218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.125144    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 00:41:33.125218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.125166    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.125218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.125214    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7htjs\""
Oct 30 00:41:33.134791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.134765    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 00:41:33.134884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.134812    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.134884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.134831    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 00:41:33.134884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.134840    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 00:41:33.135012 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.134997    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qtb46\""
Oct 30 00:41:33.135056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.135004    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 00:41:33.135056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.135024    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.135056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.135031    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-szv55\""
Oct 30 00:41:33.135155 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.135082    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.136111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136094    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.136181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136097    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 00:41:33.136181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136114    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.136181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136152    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 00:41:33.136181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136154    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-x7m5x\""
Oct 30 00:41:33.136181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136156    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.136181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136176    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.136367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.136151    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 00:41:33.158406 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.158375    4577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:33.158488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.158466    4577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:33.168562 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.168545    4577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 00:41:33.186327 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.186307    4577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:41:33.186434 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:33.186353    4577 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-17.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:33.187012 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.186999    4577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:41:33.187054 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:33.187036    4577 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-17.ec2.internal"
Oct 30 00:41:33.221469 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221442    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-run\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.221568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221478    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvl8d\" (UniqueName: \"kubernetes.io/projected/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-api-access-vvl8d\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.221667 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221648    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-cnibin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.221710 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221679    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-etc-kubernetes\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.221746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221707    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67jsn\" (UniqueName: \"kubernetes.io/projected/f50690df-0bf7-473a-a1e9-df93dacf3afa-kube-api-access-67jsn\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.221746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221735    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhjwm\" (UniqueName: \"kubernetes.io/projected/f29d1b61-cd81-4277-af77-a44af9cc5237-kube-api-access-dhjwm\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.221807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221759    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hz7\" (UniqueName: \"kubernetes.io/projected/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-kube-api-access-29hz7\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.221807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221778    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-socket-dir-parent\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.221807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221794    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-kubelet\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.221971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221816    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-netns\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.221971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221868    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-systemd\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.221971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221921    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-sys\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221966    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-tmp\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.221996    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222044    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-daemon-config\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222060    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-tmp\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222065    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-multus-certs\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222086    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-systemd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.222108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222104    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggq7r\" (UniqueName: \"kubernetes.io/projected/fbfacf74-0913-413c-b764-ec9afff26246-kube-api-access-ggq7r\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.222341 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222151    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.222341 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222185    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.222341 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222233    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-cnibin\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.222341 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222273    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4bqb\" (UniqueName: \"kubernetes.io/projected/7be17326-1cdc-43a1-b333-297b136611bf-kube-api-access-f4bqb\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.222585 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222370    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-daemon-config\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.222708 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222291    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-k8s-cni-cncf-io\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.222749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222725    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7498e2f5-ea7b-4238-9111-bde1648f2fd1-iptables-alerter-script\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.222749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222742    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b995c294-397e-41e0-8a66-6e9b225ffd34-host\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.222822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222758    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2927s\" (UniqueName: \"kubernetes.io/projected/b995c294-397e-41e0-8a66-6e9b225ffd34-kube-api-access-2927s\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.222822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222773    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f29d1b61-cd81-4277-af77-a44af9cc5237-tmp-dir\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.222822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222787    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:33.222822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222800    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9c1e69b-d783-4731-8eba-61f21232cb31-agent-certs\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:33.222822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222814    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9c1e69b-d783-4731-8eba-61f21232cb31-konnectivity-ca\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:33.223073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222903    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f29d1b61-cd81-4277-af77-a44af9cc5237-tmp-dir\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.223073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.222971    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-slash\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.223073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223004    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.223073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223033    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.223073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223059    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-env-overrides\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223089    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-kubernetes\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223113    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-host\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223139    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-device-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223147    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c9c1e69b-d783-4731-8eba-61f21232cb31-konnectivity-ca\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223165    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223168    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7498e2f5-ea7b-4238-9111-bde1648f2fd1-iptables-alerter-script\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223193    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223209    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c9c1e69b-d783-4731-8eba-61f21232cb31-agent-certs\") pod \"konnectivity-agent-2ttqh\" (UID: \"c9c1e69b-d783-4731-8eba-61f21232cb31\") " pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223239    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-cabundle\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223269    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.223298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223268    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b995c294-397e-41e0-8a66-6e9b225ffd34-serviceca\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223317    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5ws\" (UniqueName: \"kubernetes.io/projected/06eb66bd-1808-4c71-9a97-d3da60d23201-kube-api-access-xf5ws\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223325    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-metrics-tls\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223387    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-log-socket\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223414    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223419    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-cabundle\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223439    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223462    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223490    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ad924ba-4fd0-4a90-b699-01c066c8de59-crio-socket\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223535    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-kubelet-config\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223575    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-config-volume\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223591    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-env-overrides\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223595    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.223732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223691    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-systemd-units\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223789    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223836    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbfacf74-0913-413c-b764-ec9afff26246-ovn-node-metrics-cert\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223907    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-config-volume\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223921    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-modprobe-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223923    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b995c294-397e-41e0-8a66-6e9b225ffd34-serviceca\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.223986    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224007    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cqm\" (UniqueName: \"kubernetes.io/projected/2e79761e-c9f1-494b-ba28-941db904b171-kube-api-access-87cqm\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224022    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-cni-binary-copy\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224036    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-node-log\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224051    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzc66\" (UniqueName: \"kubernetes.io/projected/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-kube-api-access-tzc66\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224067    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-lib-modules\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.224225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224100    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-var-lib-kubelet\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224310    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg9gh\" (UniqueName: \"kubernetes.io/projected/87db4abb-36f5-4135-ae3a-f03cb9e2d786-kube-api-access-sg9gh\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224323    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f50690df-0bf7-473a-a1e9-df93dacf3afa-cni-binary-copy\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224338    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224363    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224394    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7498e2f5-ea7b-4238-9111-bde1648f2fd1-host-slash\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224392    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224427    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klrps\" (UniqueName: \"kubernetes.io/projected/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf-kube-api-access-klrps\") pod \"migrator-66df5d496b-k6f7p\" (UID: \"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224455    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-etc-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224477    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-ovn\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224502    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-bin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224590    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-kubelet\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224651    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.224718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224688    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-script-lib\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224723    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224757    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fbfacf74-0913-413c-b764-ec9afff26246-ovn-node-metrics-cert\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224788    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-hostroot\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224814    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zp9v\" (UniqueName: \"kubernetes.io/projected/7498e2f5-ea7b-4238-9111-bde1648f2fd1-kube-api-access-7zp9v\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224865    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224883    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06eb66bd-1808-4c71-9a97-d3da60d23201-cert\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224913    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-config\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224938    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m969p\" (UniqueName: \"kubernetes.io/projected/81f12500-d8ab-44de-bb4f-a5a987bddb69-kube-api-access-m969p\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224961    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-key\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224986    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-netns\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225009    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-multus\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.224986    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-script-lib\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225033    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f29d1b61-cd81-4277-af77-a44af9cc5237-hosts-file\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225092    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-tmp-dir\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225106    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fbfacf74-0913-413c-b764-ec9afff26246-ovnkube-config\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225111    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06eb66bd-1808-4c71-9a97-d3da60d23201-cert\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225111    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-var-lib-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225143    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-tuned\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225165    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225181    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-bin\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225184    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/87db4abb-36f5-4135-ae3a-f03cb9e2d786-signing-key\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225177    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-tmp-dir\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225195    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-netd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225223    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysconfig\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225299    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-conf\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.225311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225305    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-tuned\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225331    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225353    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-socket-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225403    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-sys-fs\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225432    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-dbus\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225463    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad924ba-4fd0-4a90-b699-01c066c8de59-data-volume\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225489    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225515    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-system-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225541    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-conf-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225565    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225629    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-system-cni-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225643    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ad924ba-4fd0-4a90-b699-01c066c8de59-data-volume\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225651    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225674    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-registration-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225693    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225797    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7be17326-1cdc-43a1-b333-297b136611bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.226256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.225807    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7088947-2eec-480b-9df3-285c4e6b2b59-original-pull-secret\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.248380 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.248355    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29hz7\" (UniqueName: \"kubernetes.io/projected/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-kube-api-access-29hz7\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.248600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.248582    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvl8d\" (UniqueName: \"kubernetes.io/projected/0ad924ba-4fd0-4a90-b699-01c066c8de59-kube-api-access-vvl8d\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.249744 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.249728    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cqm\" (UniqueName: \"kubernetes.io/projected/2e79761e-c9f1-494b-ba28-941db904b171-kube-api-access-87cqm\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.249788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.249749    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggq7r\" (UniqueName: \"kubernetes.io/projected/fbfacf74-0913-413c-b764-ec9afff26246-kube-api-access-ggq7r\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.254176 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.254161    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") pod \"periodic-gathering-blf84-qzhkv\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") " pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:41:33.254579 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.254555    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klrps\" (UniqueName: \"kubernetes.io/projected/02e80bbb-3f65-4513-8fe3-8c43c2fa83bf-kube-api-access-klrps\") pod \"migrator-66df5d496b-k6f7p\" (UID: \"02e80bbb-3f65-4513-8fe3-8c43c2fa83bf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-k6f7p"
Oct 30 00:41:33.254643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.254590    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5ws\" (UniqueName: \"kubernetes.io/projected/06eb66bd-1808-4c71-9a97-d3da60d23201-kube-api-access-xf5ws\") pod \"ingress-canary-2g945\" (UID: \"06eb66bd-1808-4c71-9a97-d3da60d23201\") " pod="openshift-ingress-canary/ingress-canary-2g945"
Oct 30 00:41:33.254946 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.254928    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzc66\" (UniqueName: \"kubernetes.io/projected/ac84b9b8-dee8-4ecf-acd0-6b093753b60c-kube-api-access-tzc66\") pod \"dns-default-hmkqx\" (UID: \"ac84b9b8-dee8-4ecf-acd0-6b093753b60c\") " pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:33.255678 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.255662    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4bqb\" (UniqueName: \"kubernetes.io/projected/7be17326-1cdc-43a1-b333-297b136611bf-kube-api-access-f4bqb\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.260143 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.260122    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67jsn\" (UniqueName: \"kubernetes.io/projected/f50690df-0bf7-473a-a1e9-df93dacf3afa-kube-api-access-67jsn\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.260210 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.260146    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhjwm\" (UniqueName: \"kubernetes.io/projected/f29d1b61-cd81-4277-af77-a44af9cc5237-kube-api-access-dhjwm\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.260266 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.260248    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgmpw\" (UniqueName: \"kubernetes.io/projected/1adb0772-ba0b-4eb7-9322-841e3ac21ebe-kube-api-access-pgmpw\") pod \"network-check-target-qtsgh\" (UID: \"1adb0772-ba0b-4eb7-9322-841e3ac21ebe\") " pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:33.260296 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.260249    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2927s\" (UniqueName: \"kubernetes.io/projected/b995c294-397e-41e0-8a66-6e9b225ffd34-kube-api-access-2927s\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.260494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.260462    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m969p\" (UniqueName: \"kubernetes.io/projected/81f12500-d8ab-44de-bb4f-a5a987bddb69-kube-api-access-m969p\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.260912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.260898    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zp9v\" (UniqueName: \"kubernetes.io/projected/7498e2f5-ea7b-4238-9111-bde1648f2fd1-kube-api-access-7zp9v\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.262771 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.262750    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg9gh\" (UniqueName: \"kubernetes.io/projected/87db4abb-36f5-4135-ae3a-f03cb9e2d786-kube-api-access-sg9gh\") pod \"service-ca-5fffc97bdf-dpp79\" (UID: \"87db4abb-36f5-4135-ae3a-f03cb9e2d786\") " pod="openshift-service-ca/service-ca-5fffc97bdf-dpp79"
Oct 30 00:41:33.326242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326174    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.326242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326207    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-cnibin\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.326242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326229    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-k8s-cni-cncf-io\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326254    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b995c294-397e-41e0-8a66-6e9b225ffd34-host\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326280    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-slash\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326299    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326301    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-cnibin\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326321    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-kubernetes\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326329    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-k8s-cni-cncf-io\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326342    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-host\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326357    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-slash\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326363    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b995c294-397e-41e0-8a66-6e9b225ffd34-host\") pod \"node-ca-5q95w\" (UID: \"b995c294-397e-41e0-8a66-6e9b225ffd34\") " pod="openshift-image-registry/node-ca-5q95w"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326376    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-device-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326410    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-host\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326365    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326424    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-kubernetes\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326456    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-device-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.326483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326466    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-log-socket\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326496    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-log-socket\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326504    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326532    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ad924ba-4fd0-4a90-b699-01c066c8de59-crio-socket\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326554    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-kubelet-config\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326574    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326580    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-systemd-units\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326605    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ad924ba-4fd0-4a90-b699-01c066c8de59-crio-socket\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326606    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326611    4577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326621    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-kubelet-config\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326622    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-systemd-units\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326637    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326642    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-modprobe-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326680    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-node-log\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326708    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-lib-modules\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326731    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-var-lib-kubelet\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326734    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-modprobe-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326734    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-node-log\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326754    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326772    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-var-lib-kubelet\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326777    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7498e2f5-ea7b-4238-9111-bde1648f2fd1-host-slash\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326800    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-etc-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326814    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-ovn\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326815    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7498e2f5-ea7b-4238-9111-bde1648f2fd1-host-slash\") pod \"iptables-alerter-clgnx\" (UID: \"7498e2f5-ea7b-4238-9111-bde1648f2fd1\") " pod="openshift-network-operator/iptables-alerter-clgnx"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326829    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326839    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-lib-modules\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326833    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-bin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326870    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-etc-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326884    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-ovn\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326889    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-bin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326897    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-kubelet\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326919    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-kubelet\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326917    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326940    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326953    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-hostroot\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326978    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-hostroot\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326979    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-netns\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.326997    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-netns\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-multus\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.327010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327021    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-cni-multus\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327035    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f29d1b61-cd81-4277-af77-a44af9cc5237-hosts-file\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327051    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-var-lib-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327065    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327079    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-bin\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327092    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-netd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327106    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysconfig\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327120    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-conf\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327125    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f29d1b61-cd81-4277-af77-a44af9cc5237-hosts-file\") pod \"node-resolver-pc8xn\" (UID: \"f29d1b61-cd81-4277-af77-a44af9cc5237\") " pod="openshift-dns/node-resolver-pc8xn"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327135    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327149    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-var-lib-openvswitch\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327175    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-socket-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327201    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-sys-fs\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327206    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327223    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-dbus\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327202    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysconfig\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327246    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-socket-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327250    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-system-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327253    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-bin\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327260    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-cni-netd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327252    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-conf\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327278    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-conf-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327287    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7088947-2eec-480b-9df3-285c4e6b2b59-dbus\") pod \"global-pull-secret-syncer-7pfth\" (UID: \"f7088947-2eec-480b-9df3-285c4e6b2b59\") " pod="kube-system/global-pull-secret-syncer-7pfth"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327295    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327289    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-system-cni-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327305    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-conf-dir\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327317    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-system-cni-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327338    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-registration-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327352    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327353    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-sys-fs\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327365    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-run\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:33.327355    4577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327387    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-cnibin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327399    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-registration-dir\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327407    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-sysctl-d\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327409    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-etc-kubernetes\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327415    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2e79761e-c9f1-494b-ba28-941db904b171-etc-selinux\") pod \"aws-ebs-csi-driver-node-pxdz4\" (UID: \"2e79761e-c9f1-494b-ba28-941db904b171\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pxdz4"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327430    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-run\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327441    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7be17326-1cdc-43a1-b333-297b136611bf-system-cni-dir\") pod \"multus-additional-cni-plugins-ms722\" (UID: \"7be17326-1cdc-43a1-b333-297b136611bf\") " pod="openshift-multus/multus-additional-cni-plugins-ms722"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:33.327452    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:33.82743235 +0000 UTC m=+1.856742578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327464    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-cnibin\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327477    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-etc-kubernetes\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327485    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-socket-dir-parent\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327508    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-kubelet\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327518    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-multus-socket-dir-parent\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327523    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-netns\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327539    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-systemd\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327553    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-sys\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327544    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-var-lib-kubelet\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327579    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-multus-certs\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327586    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-etc-systemd\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327581    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-host-run-netns\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327605    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f12500-d8ab-44de-bb4f-a5a987bddb69-sys\") pod \"tuned-57vkf\" (UID: \"81f12500-d8ab-44de-bb4f-a5a987bddb69\") " pod="openshift-cluster-node-tuning-operator/tuned-57vkf"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327603    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-systemd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327623    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f50690df-0bf7-473a-a1e9-df93dacf3afa-host-run-multus-certs\") pod \"multus-dslp8\" (UID: \"f50690df-0bf7-473a-a1e9-df93dacf3afa\") " pod="openshift-multus/multus-dslp8"
Oct 30 00:41:33.328277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.327671    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fbfacf74-0913-413c-b764-ec9afff26246-run-systemd\") pod \"ovnkube-node-xp86c\" (UID: \"fbfacf74-0913-413c-b764-ec9afff26246\") " pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:33.330649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.328837    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2907a2d-9dbd-4dae-93d6-841dfcd2a596-metrics-certs\") pod \"network-metrics-daemon-s2jdz\" (UID: \"d2907a2d-9dbd-4dae-93d6-841dfcd2a596\") " pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.393038 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.393012    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s2jdz"
Oct 30 00:41:33.395249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.395234    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:41:33.426799 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.426723    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:33.699929 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.699861    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s2jdz"]
Oct 30 00:41:33.703595 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:33.703570    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2907a2d_9dbd_4dae_93d6_841dfcd2a596.slice/crio-2d36601a829fc03901d44a9f6c966dcd2333b3df64eb3a2283fbae69f333f366 WatchSource:0}: Error finding container 2d36601a829fc03901d44a9f6c966dcd2333b3df64eb3a2283fbae69f333f366: Status 404 returned error can't find the container with id 2d36601a829fc03901d44a9f6c966dcd2333b3df64eb3a2283fbae69f333f366
Oct 30 00:41:33.832975 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:33.832937    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:33.833165 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:33.833056    4577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:33.833165 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:33.833131    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:34.833112679 +0000 UTC m=+2.862422905 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:34.164423 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.164389    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s2jdz" event={"ID":"d2907a2d-9dbd-4dae-93d6-841dfcd2a596","Type":"ContainerStarted","Data":"2d36601a829fc03901d44a9f6c966dcd2333b3df64eb3a2283fbae69f333f366"}
Oct 30 00:41:34.167369 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.167341    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hmkqx" event={"ID":"ac84b9b8-dee8-4ecf-acd0-6b093753b60c","Type":"ContainerStarted","Data":"65cdfcf4308c5b82a26295645386aa3ed97876a464542ed7a91091c30bdd12bb"}
Oct 30 00:41:34.292336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.292259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:34.478486 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.478447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:34.668918 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.668830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:34.839719 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.839677    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:34.839957 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:34.839866    4577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:34.839957 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:34.839934    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:36.839918442 +0000 UTC m=+4.869228665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:34.866916 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:34.866876    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:35.080510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:35.080467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:35.171531 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:35.171492    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s2jdz" event={"ID":"d2907a2d-9dbd-4dae-93d6-841dfcd2a596","Type":"ContainerStarted","Data":"654dc77064b171842b3c6a2feadf53c4251cc3dd4f5a47037f2d5844d6297606"}
Oct 30 00:41:35.171965 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:35.171591    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s2jdz" event={"ID":"d2907a2d-9dbd-4dae-93d6-841dfcd2a596","Type":"ContainerStarted","Data":"504dee9cb60737e75d5cc00dd180d424eef4526cf6a3979a38d951ee32ee657d"}
Oct 30 00:41:35.173225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:35.173199    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hmkqx" event={"ID":"ac84b9b8-dee8-4ecf-acd0-6b093753b60c","Type":"ContainerStarted","Data":"2ea39e347b35857479dd4c4b501ace2ea1cdc6bb6ba24ac16aa8cc6a92c81236"}
Oct 30 00:41:36.177145 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.177119    4577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:41:36.247959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.247919    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:36.266103 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.266071    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:36.471533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.471453    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:36.668568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.668541    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:36.854097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.854070    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:36.854279 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:36.854191    4577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:36.854279 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:36.854253    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls podName:0ad924ba-4fd0-4a90-b699-01c066c8de59 nodeName:}" failed. No retries permitted until 2026-10-30 00:41:40.854239227 +0000 UTC m=+8.883549449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2v4fv" (UID: "0ad924ba-4fd0-4a90-b699-01c066c8de59") : secret "insights-runtime-extractor-tls" not found
Oct 30 00:41:36.864412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:36.864385    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:37.069105 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:37.069072    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:37.649501 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:37.649474    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:37.936678 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:37.936617    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:38.060519 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.060494    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:38.183512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.183486    4577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:41:38.266886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.266862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:38.475598 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.475566    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:38.584356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.584284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:38.693306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.693252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:38.866212 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.866136    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:38.976284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:38.976255    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:39.092020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:39.091986    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:39.379758 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:39.379721    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:39.592987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:39.592957    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:40.191084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:40.191061    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:41:40.883907 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:40.883839    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:40.886251 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:40.886229    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ad924ba-4fd0-4a90-b699-01c066c8de59-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2v4fv\" (UID: \"0ad924ba-4fd0-4a90-b699-01c066c8de59\") " pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:40.894210 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:40.894190    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2v4fv"
Oct 30 00:41:41.186053 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:41.185986    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2v4fv"]
Oct 30 00:41:41.189196 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:41.189171    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad924ba_4fd0_4a90_b699_01c066c8de59.slice/crio-78dec2e430623ef07604a7232d33444c42fd30144db943ab2d5a52cf88ec496d WatchSource:0}: Error finding container 78dec2e430623ef07604a7232d33444c42fd30144db943ab2d5a52cf88ec496d: Status 404 returned error can't find the container with id 78dec2e430623ef07604a7232d33444c42fd30144db943ab2d5a52cf88ec496d
Oct 30 00:41:41.193813 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:41.193791    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2v4fv" event={"ID":"0ad924ba-4fd0-4a90-b699-01c066c8de59","Type":"ContainerStarted","Data":"78dec2e430623ef07604a7232d33444c42fd30144db943ab2d5a52cf88ec496d"}
Oct 30 00:41:41.524642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:41.524610    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:41.653202 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:41.653132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:41.766963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:41.766926    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:41.876884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:41.876840    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:42.054733 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:42.054658    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:42.198459 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:42.198429    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2v4fv" event={"ID":"0ad924ba-4fd0-4a90-b699-01c066c8de59","Type":"ContainerStarted","Data":"cf6b0ab9b2e86fe75d8eb6929af27eacb5353ad5258c98de3b600d4bf411a797"}
Oct 30 00:41:42.198864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:42.198489    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2v4fv" event={"ID":"0ad924ba-4fd0-4a90-b699-01c066c8de59","Type":"ContainerStarted","Data":"b35589435a63a89447bb025b60b84d2643895f0777cd4b1d4e5adc47fc4495fd"}
Oct 30 00:41:42.269488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:42.269458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:43.203099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:43.203068    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2v4fv" event={"ID":"0ad924ba-4fd0-4a90-b699-01c066c8de59","Type":"ContainerStarted","Data":"03f793b6d9e05dabe29eb36e761cb6e8fb5161c8b3a6a88ccf596b94d0030c51"}
Oct 30 00:41:43.526190 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:43.526154    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:43.635184 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:43.635156    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:44.003879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:44.003832    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xp86c"
Oct 30 00:41:46.068566 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.068532    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:46.179414 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.179380    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:46.182936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.182914    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hmkqx"
Oct 30 00:41:46.530186 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.530160    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-577c49dcc4-dvt9d"]
Oct 30 00:41:46.533728 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.533712    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.579899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.579861    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 00:41:46.583060 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.583031    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-577c49dcc4-dvt9d"]
Oct 30 00:41:46.609089 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.609059    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:46.615639 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.615604    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 00:41:46.615827 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.615813    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 00:41:46.615880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.615835    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6kdk2\""
Oct 30 00:41:46.615880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.615838    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 00:41:46.621810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.621792    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07421265-50a3-4221-9ba0-13605a6fe3f3-image-registry-private-configuration\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.621904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.621816    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-registry-tls\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.621904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.621836    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07421265-50a3-4221-9ba0-13605a6fe3f3-ca-trust-extracted\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.621904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.621871    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-bound-sa-token\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.622009 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.621995    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07421265-50a3-4221-9ba0-13605a6fe3f3-trusted-ca\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.622050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.622019    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07421265-50a3-4221-9ba0-13605a6fe3f3-installation-pull-secrets\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.622050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.622041    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07421265-50a3-4221-9ba0-13605a6fe3f3-registry-certificates\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.622117 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.622064    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8fnc\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-kube-api-access-p8fnc\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723327    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07421265-50a3-4221-9ba0-13605a6fe3f3-image-registry-private-configuration\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723361    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-registry-tls\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723595 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723381    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07421265-50a3-4221-9ba0-13605a6fe3f3-ca-trust-extracted\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723595 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723396    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-bound-sa-token\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723595 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723527    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07421265-50a3-4221-9ba0-13605a6fe3f3-trusted-ca\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723595 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723568    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07421265-50a3-4221-9ba0-13605a6fe3f3-installation-pull-secrets\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723600    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07421265-50a3-4221-9ba0-13605a6fe3f3-registry-certificates\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723803    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8fnc\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-kube-api-access-p8fnc\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.723949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.723821    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07421265-50a3-4221-9ba0-13605a6fe3f3-ca-trust-extracted\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.724592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.724563    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07421265-50a3-4221-9ba0-13605a6fe3f3-registry-certificates\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.724732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.724712    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07421265-50a3-4221-9ba0-13605a6fe3f3-trusted-ca\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.726048 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.726027    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-registry-tls\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.726159 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.726048    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07421265-50a3-4221-9ba0-13605a6fe3f3-image-registry-private-configuration\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.726223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.726169    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07421265-50a3-4221-9ba0-13605a6fe3f3-installation-pull-secrets\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.749472 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.749446    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-bound-sa-token\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.749648 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.749625    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8fnc\" (UniqueName: \"kubernetes.io/projected/07421265-50a3-4221-9ba0-13605a6fe3f3-kube-api-access-p8fnc\") pod \"image-registry-577c49dcc4-dvt9d\" (UID: \"07421265-50a3-4221-9ba0-13605a6fe3f3\") " pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.791981 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.791921    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:46.846215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.846192    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:46.904582 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:46.904547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:47.167032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:47.166958    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-577c49dcc4-dvt9d"]
Oct 30 00:41:47.170011 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:47.169985    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07421265_50a3_4221_9ba0_13605a6fe3f3.slice/crio-0e9ad8838d32b8f7f70a4ee1728d7b1d01f2b204cc0a5f4c298e98201f20cb87 WatchSource:0}: Error finding container 0e9ad8838d32b8f7f70a4ee1728d7b1d01f2b204cc0a5f4c298e98201f20cb87: Status 404 returned error can't find the container with id 0e9ad8838d32b8f7f70a4ee1728d7b1d01f2b204cc0a5f4c298e98201f20cb87
Oct 30 00:41:47.216707 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:47.216672    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d" event={"ID":"07421265-50a3-4221-9ba0-13605a6fe3f3","Type":"ContainerStarted","Data":"0e9ad8838d32b8f7f70a4ee1728d7b1d01f2b204cc0a5f4c298e98201f20cb87"}
Oct 30 00:41:48.221004 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:48.220968    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d" event={"ID":"07421265-50a3-4221-9ba0-13605a6fe3f3","Type":"ContainerStarted","Data":"85e69ba1e11dd9fd9f9c263e2a0a85484f21157642748632d66f2f591b6886fc"}
Oct 30 00:41:48.221602 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:48.221086    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:41:48.296793 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:48.296750    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d" podStartSLOduration=2.296735917 podStartE2EDuration="2.296735917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:48.282933517 +0000 UTC m=+16.312243749" watchObservedRunningTime="2026-10-30 00:41:48.296735917 +0000 UTC m=+16.326046146"
Oct 30 00:41:49.453390 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:49.453358    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2ttqh"
Oct 30 00:41:49.481043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:49.481016    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:49.597987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:49.597959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:49.792208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:49.792175    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:52.294166 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:52.294086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:53.977676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:53.977646    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.092625 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.092596    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.264474 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.264443    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.363794 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.363765    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"]
Oct 30 00:41:54.367088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.367069    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.392607 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.392586    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l5s6p"]
Oct 30 00:41:54.394758 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.394738    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.404868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.404833    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 00:41:54.405732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.405712    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 00:41:54.406158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.406138    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 00:41:54.406158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.406163    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 00:41:54.406711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.406692    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"]
Oct 30 00:41:54.414097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.414067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.428824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.428802    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 00:41:54.428940 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.428873    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4n894\""
Oct 30 00:41:54.428940 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.428917    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 00:41:54.428940 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.428926    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 00:41:54.429071 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.428958    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6r6tq\""
Oct 30 00:41:54.429071 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.428999    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 00:41:54.430755 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.430744    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 00:41:54.481102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.481074    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/99689af2-475f-4378-80b2-f9a655ca86fd-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.481265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.481112    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.481265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.481137    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.481265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.481223    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nz5z\" (UniqueName: \"kubernetes.io/projected/99689af2-475f-4378-80b2-f9a655ca86fd-kube-api-access-7nz5z\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.481431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.481381    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.481431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.481403    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99689af2-475f-4378-80b2-f9a655ca86fd-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.534497 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.534418    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.582706 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582678    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.582706 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582708    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nz5z\" (UniqueName: \"kubernetes.io/projected/99689af2-475f-4378-80b2-f9a655ca86fd-kube-api-access-7nz5z\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.582910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582731    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-sys\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.582910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582748    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-wtmp\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.582910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582776    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-textfile\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582953    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.583052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.582979    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-root\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583000    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99689af2-475f-4378-80b2-f9a655ca86fd-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.583192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583068    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6wbq\" (UniqueName: \"kubernetes.io/projected/0d9efebc-6819-40df-a72e-74a4c9766174-kube-api-access-f6wbq\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583114    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/99689af2-475f-4378-80b2-f9a655ca86fd-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.583192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583166    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583186    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-accelerators-collector-config\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583277    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.583373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583315    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-tls\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583349    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9efebc-6819-40df-a72e-74a4c9766174-metrics-client-ca\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.583509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583429    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.583509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583459    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/99689af2-475f-4378-80b2-f9a655ca86fd-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.583793 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.583769    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99689af2-475f-4378-80b2-f9a655ca86fd-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.585536 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.585515    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.585612 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.585561    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99689af2-475f-4378-80b2-f9a655ca86fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.607257 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.607236    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nz5z\" (UniqueName: \"kubernetes.io/projected/99689af2-475f-4378-80b2-f9a655ca86fd-kube-api-access-7nz5z\") pod \"kube-state-metrics-5dc8d7d89b-prwfk\" (UID: \"99689af2-475f-4378-80b2-f9a655ca86fd\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.646384 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.646360    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.679989 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.679963    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"
Oct 30 00:41:54.684396 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684379    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-tls\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684410    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9efebc-6819-40df-a72e-74a4c9766174-metrics-client-ca\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684444    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-sys\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684469    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-wtmp\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684495    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-textfile\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684535    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-root\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684564    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6wbq\" (UniqueName: \"kubernetes.io/projected/0d9efebc-6819-40df-a72e-74a4c9766174-kube-api-access-f6wbq\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684598    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684617    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-sys\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684623    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-accelerators-collector-config\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684671    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-wtmp\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.684949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684676    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0d9efebc-6819-40df-a72e-74a4c9766174-root\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.685037 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.684958    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-textfile\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.685069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.685040    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9efebc-6819-40df-a72e-74a4c9766174-metrics-client-ca\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.685208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.685186    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-accelerators-collector-config\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.687009 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.686961    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-tls\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.687144 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.687128    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9efebc-6819-40df-a72e-74a4c9766174-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.709273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.709243    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6wbq\" (UniqueName: \"kubernetes.io/projected/0d9efebc-6819-40df-a72e-74a4c9766174-kube-api-access-f6wbq\") pod \"node-exporter-l5s6p\" (UID: \"0d9efebc-6819-40df-a72e-74a4c9766174\") " pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:54.758635 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.758604    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:54.988228 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:54.988127    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk"]
Oct 30 00:41:54.990726 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:54.990706    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99689af2_475f_4378_80b2_f9a655ca86fd.slice/crio-ee23b5b634eb3e416252d2b5d2d01af0f890447833a1a0ac438bb0a789ad610a WatchSource:0}: Error finding container ee23b5b634eb3e416252d2b5d2d01af0f890447833a1a0ac438bb0a789ad610a: Status 404 returned error can't find the container with id ee23b5b634eb3e416252d2b5d2d01af0f890447833a1a0ac438bb0a789ad610a
Oct 30 00:41:55.006384 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.006365    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l5s6p"
Oct 30 00:41:55.015953 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:41:55.015927    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9efebc_6819_40df_a72e_74a4c9766174.slice/crio-fe07cd3d3c73db0e5f32d002236323a360a0936b4566b56867f3d2a1a76f422c WatchSource:0}: Error finding container fe07cd3d3c73db0e5f32d002236323a360a0936b4566b56867f3d2a1a76f422c: Status 404 returned error can't find the container with id fe07cd3d3c73db0e5f32d002236323a360a0936b4566b56867f3d2a1a76f422c
Oct 30 00:41:55.250499 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.250407    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l5s6p" event={"ID":"0d9efebc-6819-40df-a72e-74a4c9766174","Type":"ContainerStarted","Data":"fe07cd3d3c73db0e5f32d002236323a360a0936b4566b56867f3d2a1a76f422c"}
Oct 30 00:41:55.251589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.251562    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk" event={"ID":"99689af2-475f-4378-80b2-f9a655ca86fd","Type":"ContainerStarted","Data":"ee23b5b634eb3e416252d2b5d2d01af0f890447833a1a0ac438bb0a789ad610a"}
Oct 30 00:41:55.433621 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.433537    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:55.551896 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.551792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:55.666029 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.665993    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:55.776247 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:55.776203    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:56.256861 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:56.256822    4577 generic.go:356] "Generic (PLEG): container finished" podID="0d9efebc-6819-40df-a72e-74a4c9766174" containerID="3617f2d6dafd3f884304671843f620dc6c37dc8da32ba465f9b71390029bf5fe" exitCode=0
Oct 30 00:41:56.257468 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:56.257442    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l5s6p" event={"ID":"0d9efebc-6819-40df-a72e-74a4c9766174","Type":"ContainerDied","Data":"3617f2d6dafd3f884304671843f620dc6c37dc8da32ba465f9b71390029bf5fe"}
Oct 30 00:41:56.259448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:56.259422    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk" event={"ID":"99689af2-475f-4378-80b2-f9a655ca86fd","Type":"ContainerStarted","Data":"7bcc63b0e8bbf27a57f245b10c25eb357e40d788a09a996098acd6271253a449"}
Oct 30 00:41:56.259568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:56.259473    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk" event={"ID":"99689af2-475f-4378-80b2-f9a655ca86fd","Type":"ContainerStarted","Data":"294e5c860f665f667e859ee5bf2bcbbc9202a6a5ba794441c0d03a5905845c62"}
Oct 30 00:41:57.011518 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:41:57.011486    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:41:57.265353 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.265242    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l5s6p" event={"ID":"0d9efebc-6819-40df-a72e-74a4c9766174","Type":"ContainerStarted","Data":"d05d3c9f545c2052f78f613989bab480d67bcfeb9211aa9fee1fd8bb2e8e5a29"}
Oct 30 00:41:57.265778 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.265358    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l5s6p" event={"ID":"0d9efebc-6819-40df-a72e-74a4c9766174","Type":"ContainerStarted","Data":"4c476b67f9c2f3fd8e509eec411fb3837ffa04a723fb829a83eda8cd06a4970f"}
Oct 30 00:41:57.267488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.267470    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk" event={"ID":"99689af2-475f-4378-80b2-f9a655ca86fd","Type":"ContainerStarted","Data":"fe1a0c54ecf0db727f787a153dfeef00d36cc5b675c5269087c848bab3138144"}
Oct 30 00:41:57.329992 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.329959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:57.342372 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.342325    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l5s6p" podStartSLOduration=2.352247844 podStartE2EDuration="3.342313288s" totalImagesPullingTime="907.115044ms" totalInitContainerRuntime="82.9504ms" isStatefulPod=true podCreationTimestamp="2026-10-30 00:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:57.328128114 +0000 UTC m=+25.357438343" watchObservedRunningTime="2026-10-30 00:41:57.342313288 +0000 UTC m=+25.371623516"
Oct 30 00:41:57.374228 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.374173    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-prwfk" podStartSLOduration=2.228770153 podStartE2EDuration="3.374161595s" totalImagesPullingTime="1.145391442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:41:57.374056019 +0000 UTC m=+25.403366250" watchObservedRunningTime="2026-10-30 00:41:57.374161595 +0000 UTC m=+25.403471817"
Oct 30 00:41:57.438493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.438470    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:57.553054 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.552980    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:57.669085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:57.669059    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:59.062481 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.062450    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:59.183481 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.183447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:59.300623 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.300572    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:41:59.713122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.713083    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"]
Oct 30 00:41:59.746721 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.746688    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"]
Oct 30 00:41:59.746907 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.746810    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.766310 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.766288    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 00:41:59.766458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.766322    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 00:41:59.766732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.766706    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 00:41:59.766732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.766714    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 00:41:59.766936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.766789    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 00:41:59.780383 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.780364    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-n25lv\""
Oct 30 00:41:59.794592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.794574    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 00:41:59.818957 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.818931    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-metrics-client-ca\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.818967    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-secret-telemeter-client\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.818993    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-federate-client-tls\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819167 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.819125    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819167 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.819148    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bshd\" (UniqueName: \"kubernetes.io/projected/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-kube-api-access-2bshd\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819247 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.819182    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-serving-certs-ca-bundle\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819292 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.819278    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-telemeter-client-tls\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.819346 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.819311    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.920528    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.920603    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bshd\" (UniqueName: \"kubernetes.io/projected/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-kube-api-access-2bshd\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.920652    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-serving-certs-ca-bundle\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.920881    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-telemeter-client-tls\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.920915    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.921001    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-metrics-client-ca\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.921045    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-secret-telemeter-client\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.924322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.921074    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-federate-client-tls\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.925581 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.925166    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-federate-client-tls\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.925581 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.925526    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-metrics-client-ca\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.925742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.925651    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-serving-certs-ca-bundle\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.925742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.925720    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.927033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.927013    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-telemeter-client-tls\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.927570 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.927549    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.927643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.927576    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-secret-telemeter-client\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:41:59.949709 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:41:59.949682    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bshd\" (UniqueName: \"kubernetes.io/projected/93128ccb-ef11-4510-8f32-9e7e0c33a6fd-kube-api-access-2bshd\") pod \"telemeter-client-7d9f98946d-bhq9t\" (UID: \"93128ccb-ef11-4510-8f32-9e7e0c33a6fd\") " pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:42:00.094219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:00.094192    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"
Oct 30 00:42:00.193988 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:00.193961    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qtsgh"
Oct 30 00:42:00.453533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:00.453464    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d9f98946d-bhq9t"]
Oct 30 00:42:00.469139 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:42:00.469107    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93128ccb_ef11_4510_8f32_9e7e0c33a6fd.slice/crio-9301fe3c492b7b634cee375e9bdf85a889020c5dec163eff0b3cc5988ee4674a WatchSource:0}: Error finding container 9301fe3c492b7b634cee375e9bdf85a889020c5dec163eff0b3cc5988ee4674a: Status 404 returned error can't find the container with id 9301fe3c492b7b634cee375e9bdf85a889020c5dec163eff0b3cc5988ee4674a
Oct 30 00:42:01.284011 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.283977    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t" event={"ID":"93128ccb-ef11-4510-8f32-9e7e0c33a6fd","Type":"ContainerStarted","Data":"9301fe3c492b7b634cee375e9bdf85a889020c5dec163eff0b3cc5988ee4674a"}
Oct 30 00:42:01.395460 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.395434    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dc4f77f49-28g49"]
Oct 30 00:42:01.395730 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.395709    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:01.416933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.416908    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.433990 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.433956    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434126 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.433998    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmb98\" (UniqueName: \"kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434126 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.434027    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434126 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.434094    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434126 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.434116    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434286 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.434145    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434286 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.434159    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.434286 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.434176    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.440184 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.440157    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 00:42:01.440338 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.440271    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 00:42:01.440409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.440390    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 00:42:01.440499 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.440477    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 00:42:01.446623 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.446583    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 00:42:01.447420 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.447395    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc4f77f49-28g49"]
Oct 30 00:42:01.468707 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.468684    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 00:42:01.498405 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.498382    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 00:42:01.498693 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.498682    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7qn49\""
Oct 30 00:42:01.498756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.498689    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 00:42:01.535381 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535305    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535381 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535362    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535384    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmb98\" (UniqueName: \"kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535411    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535440    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535465    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535502    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535526    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.535902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.535789    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.536300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.536275    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.536632 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.536616    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.537252 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.537230    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.537630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.537614    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.538237 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.538217    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.538506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.538486    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.561408 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.561386    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmb98\" (UniqueName: \"kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98\") pod \"console-5dc4f77f49-28g49\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") " pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:01.565002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.564981    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:01.741963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:01.741933    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:02.063710 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:02.063679    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc4f77f49-28g49"]
Oct 30 00:42:02.067389 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:42:02.067360    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f67bf32_14c0_49dc_b4e9_a9dbb40927ec.slice/crio-f35383b56b897040d0042b993b5e7f93263daae62231f2036d7695f347b18dbd WatchSource:0}: Error finding container f35383b56b897040d0042b993b5e7f93263daae62231f2036d7695f347b18dbd: Status 404 returned error can't find the container with id f35383b56b897040d0042b993b5e7f93263daae62231f2036d7695f347b18dbd
Oct 30 00:42:02.287830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:02.287789    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc4f77f49-28g49" event={"ID":"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec","Type":"ContainerStarted","Data":"f35383b56b897040d0042b993b5e7f93263daae62231f2036d7695f347b18dbd"}
Oct 30 00:42:03.294708 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:03.294672    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t" event={"ID":"93128ccb-ef11-4510-8f32-9e7e0c33a6fd","Type":"ContainerStarted","Data":"8cb20e46d3d5fb1c3ecf8202652348f65d774aeda0567a3d205d0eb594a665b9"}
Oct 30 00:42:03.546529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:03.546371    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:03.669219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:03.669187    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:04.303969 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:04.303926    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t" event={"ID":"93128ccb-ef11-4510-8f32-9e7e0c33a6fd","Type":"ContainerStarted","Data":"27ae2c64474e2b8f54fe6954a3a55e3f47cd1eb03b1b9a0da58c3d2fe8a181be"}
Oct 30 00:42:04.304387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:04.303988    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t" event={"ID":"93128ccb-ef11-4510-8f32-9e7e0c33a6fd","Type":"ContainerStarted","Data":"df3f6cd5490914020bd4fc93d82b79d41e9868d9c6918fe112c0b14ec9f79137"}
Oct 30 00:42:04.383748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:04.383644    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d9f98946d-bhq9t" podStartSLOduration=1.894687885 podStartE2EDuration="5.383625008s" totalImagesPullingTime="3.488937123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:41:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:42:04.369295064 +0000 UTC m=+32.398605293" watchObservedRunningTime="2026-10-30 00:42:04.383625008 +0000 UTC m=+32.412935239"
Oct 30 00:42:05.172725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:05.172691    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:05.284415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:05.284387    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:05.744111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:05.744080    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:05.779792 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:05.779732    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc4f77f49-28g49"]
Oct 30 00:42:05.862451 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:05.862408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:06.315290 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:06.315257    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc4f77f49-28g49" event={"ID":"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec","Type":"ContainerStarted","Data":"5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3"}
Oct 30 00:42:06.361881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:06.361815    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5dc4f77f49-28g49" podStartSLOduration=1.9222342110000001 podStartE2EDuration="5.361800831s" totalImagesPullingTime="3.43956662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:42:06.361775868 +0000 UTC m=+34.391086103" watchObservedRunningTime="2026-10-30 00:42:06.361800831 +0000 UTC m=+34.391111060"
Oct 30 00:42:07.249933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:07.249900    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:07.368539 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:07.368505    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:07.938457 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:07.938412    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:08.050648 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:08.050619    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:09.201471 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:09.201435    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:09.228409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:09.228384    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-577c49dcc4-dvt9d"
Oct 30 00:42:09.319104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:09.319069    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:09.447632 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:09.447598    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:09.562282 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:09.562250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:09.677045 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:09.677013    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:10.192888 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:10.192860    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:10.301541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:10.301511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:11.742880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:11.742836    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:14.334091 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:14.334017    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:14.441524 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:14.441491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:16.072547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:16.072503    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:16.181673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:16.181629    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:17.594389 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:17.594355    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:17.732048 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:17.732012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:27.011878 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:42:27.011825    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:42:29.832824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:29.832783    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:29.949826 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:29.949792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:30.060158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:30.060121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:31.064662 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:31.064622    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:31.173647 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:31.173617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:32.338543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.338490    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5dc4f77f49-28g49" podUID="9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" containerName="console" containerID="cri-o://5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3" gracePeriod=15
Oct 30 00:42:32.367925 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:42:32.367885    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f67bf32_14c0_49dc_b4e9_a9dbb40927ec.slice/crio-5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f67bf32_14c0_49dc_b4e9_a9dbb40927ec.slice/crio-conmon-5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 00:42:32.587230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.583619    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:32.676363 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676277    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676363 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676342    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676396    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98\" (UniqueName: \"kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676441    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676465    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676496    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676523    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676572    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle\") pod \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\" (UID: \"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec\") "
Oct 30 00:42:32.676816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676596    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:42:32.676816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676760    4577 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-tmp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.676816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.676712    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:42:32.677072 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.677014    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:42:32.677072 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.677033    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:42:32.677205 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.677162    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:42:32.679022 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.678991    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:42:32.679022 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.679004    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:42:32.679142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.679024    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98" pod "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" (UID: "9f67bf32-14c0-49dc-b4e9-a9dbb40927ec"). InnerVolumeSpecName "kube-api-access-pmb98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:42:32.777819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777786    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmb98\" (UniqueName: \"kubernetes.io/projected/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-kube-api-access-pmb98\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.777819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777814    4577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-oauth-config\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.777819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777824    4577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-config\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.778058 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777833    4577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-oauth-serving-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.778058 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777860    4577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-console-serving-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.778058 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777869    4577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-trusted-ca-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:32.778058 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:32.777876    4577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:33.402737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.402699    4577 generic.go:356] "Generic (PLEG): container finished" podID="9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" containerID="5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3" exitCode=0
Oct 30 00:42:33.402737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.402737    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc4f77f49-28g49" event={"ID":"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec","Type":"ContainerDied","Data":"5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3"}
Oct 30 00:42:33.403302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.402763    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc4f77f49-28g49"
Oct 30 00:42:33.403302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.402786    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc4f77f49-28g49" event={"ID":"9f67bf32-14c0-49dc-b4e9-a9dbb40927ec","Type":"ContainerDied","Data":"f35383b56b897040d0042b993b5e7f93263daae62231f2036d7695f347b18dbd"}
Oct 30 00:42:33.403302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.402806    4577 scope.go:122] "RemoveContainer" containerID="5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3"
Oct 30 00:42:33.411392 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.411374    4577 scope.go:122] "RemoveContainer" containerID="5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3"
Oct 30 00:42:33.411675 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:42:33.411655    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3\": container with ID starting with 5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3 not found: ID does not exist" containerID="5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3"
Oct 30 00:42:33.411722 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.411686    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3"} err="failed to get container status \"5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3\": rpc error: code = NotFound desc = could not find container \"5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3\": container with ID starting with 5cb4bf4dd226f0398a4505ad09f5347250c0a9dec4371b53d3d6a6b2bf3410a3 not found: ID does not exist"
Oct 30 00:42:33.462767 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.462734    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc4f77f49-28g49"]
Oct 30 00:42:33.467042 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:33.467022    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dc4f77f49-28g49"]
Oct 30 00:42:34.108050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:34.108023    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" path="/var/lib/kubelet/pods/9f67bf32-14c0-49dc-b4e9-a9dbb40927ec/volumes"
Oct 30 00:42:40.816065 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:40.816033    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:40.924112 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:40.924075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:49.459595 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:49.459563    4577 generic.go:356] "Generic (PLEG): container finished" podID="490ed73e-84fe-4053-bec4-43f479935b3d" containerID="03faec73fa1a03bbbb1c32e5dc95af0cdc84b3d3e061bdcea9e141f0e113e860" exitCode=0
Oct 30 00:42:49.460112 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:49.459626    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-blf84-qzhkv" event={"ID":"490ed73e-84fe-4053-bec4-43f479935b3d","Type":"ContainerDied","Data":"03faec73fa1a03bbbb1c32e5dc95af0cdc84b3d3e061bdcea9e141f0e113e860"}
Oct 30 00:42:50.595099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.595076    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:42:50.708781 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.708744    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") pod \"490ed73e-84fe-4053-bec4-43f479935b3d\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") "
Oct 30 00:42:50.708966 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.708792    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") pod \"490ed73e-84fe-4053-bec4-43f479935b3d\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") "
Oct 30 00:42:50.708966 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.708876    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") pod \"490ed73e-84fe-4053-bec4-43f479935b3d\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") "
Oct 30 00:42:50.708966 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.708924    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") pod \"490ed73e-84fe-4053-bec4-43f479935b3d\" (UID: \"490ed73e-84fe-4053-bec4-43f479935b3d\") "
Oct 30 00:42:50.709200 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.709182    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle" pod "490ed73e-84fe-4053-bec4-43f479935b3d" (UID: "490ed73e-84fe-4053-bec4-43f479935b3d"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:42:50.709236 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.709131    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path" pod "490ed73e-84fe-4053-bec4-43f479935b3d" (UID: "490ed73e-84fe-4053-bec4-43f479935b3d"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:42:50.709328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.709304    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle" pod "490ed73e-84fe-4053-bec4-43f479935b3d" (UID: "490ed73e-84fe-4053-bec4-43f479935b3d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:42:50.711373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.711354    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs" pod "490ed73e-84fe-4053-bec4-43f479935b3d" (UID: "490ed73e-84fe-4053-bec4-43f479935b3d"). InnerVolumeSpecName "kube-api-access-x7czs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:42:50.810336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.810280    4577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-trusted-ca-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:50.810336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.810329    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7czs\" (UniqueName: \"kubernetes.io/projected/490ed73e-84fe-4053-bec4-43f479935b3d-kube-api-access-x7czs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:50.810336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.810342    4577 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/490ed73e-84fe-4053-bec4-43f479935b3d-archives-path\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:50.810579 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:50.810354    4577 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490ed73e-84fe-4053-bec4-43f479935b3d-service-ca-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:42:51.156764 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:51.156728    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:51.270086 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:51.270057    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:51.467813 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:51.467747    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-blf84-qzhkv"
Oct 30 00:42:51.467813 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:51.467755    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-blf84-qzhkv" event={"ID":"490ed73e-84fe-4053-bec4-43f479935b3d","Type":"ContainerDied","Data":"56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684"}
Oct 30 00:42:51.467813 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:51.467798    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56a716aa989186aa2d0a0b8f49cc3f709c4b5907959ef1f60ef554374d84d684"
Oct 30 00:42:51.955702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:51.955672    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:52.064023 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:52.063986    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:42:57.011867 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:42:57.011811    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:42:57.996482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996441    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:42:57.996778 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996762    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" containerName="console"
Oct 30 00:42:57.996879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996780    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" containerName="console"
Oct 30 00:42:57.996879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996805    4577 cpu_manager.go:414] "containerMap: removing container" podUID="490ed73e-84fe-4053-bec4-43f479935b3d" containerName="insights-gathering"
Oct 30 00:42:57.996879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996813    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="490ed73e-84fe-4053-bec4-43f479935b3d" containerName="insights-gathering"
Oct 30 00:42:57.997032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996892    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="490ed73e-84fe-4053-bec4-43f479935b3d" containerName="insights-gathering"
Oct 30 00:42:57.997032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:57.996908    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f67bf32-14c0-49dc-b4e9-a9dbb40927ec" containerName="console"
Oct 30 00:42:58.001070 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.001049    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.013225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.013203    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026339    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026356    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k2z8g\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026377    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026356    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026397    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026351    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026339    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 00:42:58.026427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026405    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 00:42:58.026784 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.026443    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 00:42:58.054471 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.054449    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 00:42:58.160903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.160871    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.160903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.160906    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161146 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.160923    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss4hl\" (UniqueName: \"kubernetes.io/projected/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-kube-api-access-ss4hl\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161146 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161004    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-web-config\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161146 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161028    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161146 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161103    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161193    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161221    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161250    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161277    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161346    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161414    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-config-out\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.161530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.161433    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.262777 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.262693    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.262777 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.262729    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.262777 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.262759    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.262901    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.262942    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.262969    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263001    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263036    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-config-out\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263062    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263366 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263100    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263366 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263135    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263366 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263163    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss4hl\" (UniqueName: \"kubernetes.io/projected/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-kube-api-access-ss4hl\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263366 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263189    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-web-config\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.263927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.263902    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.264220 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.264198    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.264476 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.264450    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.266220 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.265953    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.266220 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.266023    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-web-config\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.266220 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.266120    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.266220 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.266166    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.266864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.266826    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.267021 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.266997    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.267564 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.267547    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.267988 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.267956    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.268108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.268090    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-config-out\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.291344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.291325    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss4hl\" (UniqueName: \"kubernetes.io/projected/d7f8ea78-d9c6-4774-8938-0e273c1e2ad1-kube-api-access-ss4hl\") pod \"alertmanager-main-0\" (UID: \"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.324554 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.324529    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:42:58.654008 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:58.653985    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:42:58.656228 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:42:58.656204    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f8ea78_d9c6_4774_8938_0e273c1e2ad1.slice/crio-2241f829c2de565951989a809f543b2e128e3f4ead39c336f4dbbb298c2e3c0a WatchSource:0}: Error finding container 2241f829c2de565951989a809f543b2e128e3f4ead39c336f4dbbb298c2e3c0a: Status 404 returned error can't find the container with id 2241f829c2de565951989a809f543b2e128e3f4ead39c336f4dbbb298c2e3c0a
Oct 30 00:42:59.494570 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:59.494537    4577 generic.go:356] "Generic (PLEG): container finished" podID="d7f8ea78-d9c6-4774-8938-0e273c1e2ad1" containerID="46fcd05cafa2c074edbe2d38b71e6bee5aaa611e8d325fcaf6437366deb66d85" exitCode=0
Oct 30 00:42:59.495106 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:59.494622    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerDied","Data":"46fcd05cafa2c074edbe2d38b71e6bee5aaa611e8d325fcaf6437366deb66d85"}
Oct 30 00:42:59.495106 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:42:59.494712    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"2241f829c2de565951989a809f543b2e128e3f4ead39c336f4dbbb298c2e3c0a"}
Oct 30 00:43:01.503809 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:01.503772    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"a61b67ce1993c59816bb88b5e9da84b769969c2c6328f27b6d23fd1dc9b16236"}
Oct 30 00:43:01.504214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:01.503831    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"f92ba33e31bd034a85cecef8c5294dad8c95cc01c553c384f41d594e98f678d0"}
Oct 30 00:43:01.504214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:01.503877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"d95a4125bf01a44eca484e5ac4a7b9c3c542d9cd94595cf3d45503058944535c"}
Oct 30 00:43:01.504214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:01.503899    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"4c6181d22695309619bbc194b6d5471dc443d2b1f30b8874cd7f5f4c9a086579"}
Oct 30 00:43:01.504214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:01.503921    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"01c41683335607b42c71e21be88e1492e582ae1fdebcb04628cff2f1811b7cfa"}
Oct 30 00:43:02.510635 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:02.510596    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7f8ea78-d9c6-4774-8938-0e273c1e2ad1","Type":"ContainerStarted","Data":"fc25addd46052b200a9b985f860f188c72581c71daa053b9c7aa0d3e6faad8f5"}
Oct 30 00:43:02.607741 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:02.607685    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.511174556 podStartE2EDuration="5.607671652s" totalImagesPullingTime="2.085474145s" totalInitContainerRuntime="11.022951ms" isStatefulPod=false podCreationTimestamp="2026-10-30 00:42:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:02.607062499 +0000 UTC m=+90.636372728" watchObservedRunningTime="2026-10-30 00:43:02.607671652 +0000 UTC m=+90.636981881"
Oct 30 00:43:04.102573 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.102544    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b76f84fd4-lqkj8"]
Oct 30 00:43:04.105854 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.105823    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.112495 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112469    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-oauth-serving-cert\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.112631 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112517    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-oauth-config\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.112631 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112616    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-service-ca\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.113020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112683    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmwtk\" (UniqueName: \"kubernetes.io/projected/7f0f01d3-84a1-4200-993f-ab77f7b90113-kube-api-access-qmwtk\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.113020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112804    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-config\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.113020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112901    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-serving-cert\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.113020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.112968    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f0f01d3-84a1-4200-993f-ab77f7b90113-tmp\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.113020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.113000    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-trusted-ca-bundle\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.125808 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.125779    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b76f84fd4-lqkj8"]
Oct 30 00:43:04.128308 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.128278    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 00:43:04.128432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.128315    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 00:43:04.128432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.128319    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 00:43:04.128533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.128485    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 00:43:04.128533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.128521    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 00:43:04.128622 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.128554    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 00:43:04.138518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.138496    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7qn49\""
Oct 30 00:43:04.138518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.138515    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 00:43:04.156642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.156593    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 00:43:04.213652 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.213618    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-oauth-config\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.213862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.213664    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-service-ca\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.213862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.213692    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmwtk\" (UniqueName: \"kubernetes.io/projected/7f0f01d3-84a1-4200-993f-ab77f7b90113-kube-api-access-qmwtk\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.213976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.213880    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-config\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.213976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.213922    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-serving-cert\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.213976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.213941    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f0f01d3-84a1-4200-993f-ab77f7b90113-tmp\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.214150 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.214127    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-trusted-ca-bundle\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.214212 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.214199    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-oauth-serving-cert\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.214265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.214249    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f0f01d3-84a1-4200-993f-ab77f7b90113-tmp\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.214566 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.214539    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-service-ca\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.214807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.214786    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-config\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.214874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.214794    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-oauth-serving-cert\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.215331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.215310    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f0f01d3-84a1-4200-993f-ab77f7b90113-trusted-ca-bundle\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.216429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.216407    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-serving-cert\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.216504 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.216434    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f0f01d3-84a1-4200-993f-ab77f7b90113-console-oauth-config\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.240154 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.240126    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmwtk\" (UniqueName: \"kubernetes.io/projected/7f0f01d3-84a1-4200-993f-ab77f7b90113-kube-api-access-qmwtk\") pod \"console-7b76f84fd4-lqkj8\" (UID: \"7f0f01d3-84a1-4200-993f-ab77f7b90113\") " pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.288306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.288277    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:04.402322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.402252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:04.418466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.418437    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:04.546536 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.546490    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:04.659266 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.659174    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:04.714877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:04.714741    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b76f84fd4-lqkj8"]
Oct 30 00:43:04.717421 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:43:04.717374    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0f01d3_84a1_4200_993f_ab77f7b90113.slice/crio-d18e9d6a81f1843712cd3cdc7ba1cb78cc48e4ed499fba3e12a5ceef42bfe28e WatchSource:0}: Error finding container d18e9d6a81f1843712cd3cdc7ba1cb78cc48e4ed499fba3e12a5ceef42bfe28e: Status 404 returned error can't find the container with id d18e9d6a81f1843712cd3cdc7ba1cb78cc48e4ed499fba3e12a5ceef42bfe28e
Oct 30 00:43:05.522067 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:05.522032    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b76f84fd4-lqkj8" event={"ID":"7f0f01d3-84a1-4200-993f-ab77f7b90113","Type":"ContainerStarted","Data":"cbc54a08e255d16d4c3db90d7a324d3fd161b0ffaf7d47e93d9e8928b16d13e4"}
Oct 30 00:43:05.522582 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:05.522081    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b76f84fd4-lqkj8" event={"ID":"7f0f01d3-84a1-4200-993f-ab77f7b90113","Type":"ContainerStarted","Data":"d18e9d6a81f1843712cd3cdc7ba1cb78cc48e4ed499fba3e12a5ceef42bfe28e"}
Oct 30 00:43:08.095317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:08.095271    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:08.202649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:08.202607    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:14.419221 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:14.419163    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:14.419221 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:14.419235    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:14.424210 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:14.424184    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:14.477340 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:14.477286    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7b76f84fd4-lqkj8" podStartSLOduration=10.477270207 podStartE2EDuration="10.477270207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:05.582000674 +0000 UTC m=+93.611310905" watchObservedRunningTime="2026-10-30 00:43:14.477270207 +0000 UTC m=+102.506580436"
Oct 30 00:43:14.562168 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:14.562053    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b76f84fd4-lqkj8"
Oct 30 00:43:16.072530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:16.072499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:16.181014 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:16.180980    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:17.704317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:17.704275    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:27.011565 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:43:27.011528    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:43:38.701705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:38.701670    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:38.812803 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:43:38.812775    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:43:57.012276 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:43:57.012235    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:44:17.833105 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:44:17.833067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:44:27.012087 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:44:27.012051    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:44:54.016713 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:44:54.016677    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:44:57.011579 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:44:57.011536    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:45:03.318905 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:03.318869    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:17.953634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:17.953561    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:27.012379 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:45:27.012343    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:45:34.292399 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:34.292366    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:34.483430 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:34.483401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:35.801097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:35.801059    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:35.946602 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:35.946571    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:37.121192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:37.121157    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:38.069399 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.069357    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"]
Oct 30 00:45:38.072811 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.072788    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.106082 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.106056    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 00:45:38.106082 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.106059    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 00:45:38.106437 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.106419    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-qpslw\""
Oct 30 00:45:38.108338 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.108321    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"]
Oct 30 00:45:38.217971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.217934    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.218418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.218093    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmctj\" (UniqueName: \"kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.218418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.218154    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.319332 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.319291    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmctj\" (UniqueName: \"kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.319560 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.319344    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.319560 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.319385    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.319751 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.319727    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.319823 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.319775    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.347557 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.347526    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmctj\" (UniqueName: \"kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.387991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.387963    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:45:38.685719 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:38.685640    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"]
Oct 30 00:45:38.688712 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:38.688685    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54acdd8a_5fe1_415d_b407_551e9937e971.slice/crio-62ac07892eb1936fe658465ec38871afd6e54c6fc09bd083dae61bb637798c61 WatchSource:0}: Error finding container 62ac07892eb1936fe658465ec38871afd6e54c6fc09bd083dae61bb637798c61: Status 404 returned error can't find the container with id 62ac07892eb1936fe658465ec38871afd6e54c6fc09bd083dae61bb637798c61
Oct 30 00:45:39.037058 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.037024    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl" event={"ID":"54acdd8a-5fe1-415d-b407-551e9937e971","Type":"ContainerStarted","Data":"62ac07892eb1936fe658465ec38871afd6e54c6fc09bd083dae61bb637798c61"}
Oct 30 00:45:39.337092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.337014    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"]
Oct 30 00:45:39.340737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.340713    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.372027 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.371992    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-xqwv2\""
Oct 30 00:45:39.373560 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.373536    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"]
Oct 30 00:45:39.428404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.428372    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.428404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.428407    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.428643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.428535    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbkv7\" (UniqueName: \"kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.443342 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.443305    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"]
Oct 30 00:45:39.446922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.446900    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.466638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.466605    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"]
Oct 30 00:45:39.467491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.467452    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-xfvxp\""
Oct 30 00:45:39.529686 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.529646    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbkv7\" (UniqueName: \"kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.529952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.529719    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.529952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.529748    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqwh\" (UniqueName: \"kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.529952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.529797    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.529952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.529929    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.530194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.530011    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.530249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.530206    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.530308 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.530292    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.539899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.539866    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"]
Oct 30 00:45:39.543733 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.543712    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.556186 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.556161    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbkv7\" (UniqueName: \"kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.563162 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.563135    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-kpclg\""
Oct 30 00:45:39.564836 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.564811    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"]
Oct 30 00:45:39.631108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631027    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.631108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631074    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.631349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631130    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.631349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631154    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqwh\" (UniqueName: \"kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.631349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631189    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.631349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631259    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlt9p\" (UniqueName: \"kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.631539 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631476    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.631539 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.631509    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.640988 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.640962    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"]
Oct 30 00:45:39.645207 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.645185    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.656773 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.656751    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqwh\" (UniqueName: \"kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.658394 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.658364    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:45:39.662141 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.662115    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-2jj87\""
Oct 30 00:45:39.663713 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.663665    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"]
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.734062    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.734108    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlt9p\" (UniqueName: \"kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.734143    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.734598    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.734683    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.734716    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppqd7\" (UniqueName: \"kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.735220    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.736589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.735235    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.759722 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.759688    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlt9p\" (UniqueName: \"kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.764655 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.764632    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:45:39.835646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.835619    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppqd7\" (UniqueName: \"kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.835750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.835660    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.835750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.835689    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.836088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.836071    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.836543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.836527    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.863960 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.863933    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppqd7\" (UniqueName: \"kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.869648 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.869627    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:45:39.965937 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.965907    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:45:39.970967 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:39.970932    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"]
Oct 30 00:45:39.975118 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:39.975083    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343d735a_b95a_447f_869a_6cb18af8f934.slice/crio-931e214f14f092bbf29413c57edba04c302573091ee3ae8d16f0f3ca01772169 WatchSource:0}: Error finding container 931e214f14f092bbf29413c57edba04c302573091ee3ae8d16f0f3ca01772169: Status 404 returned error can't find the container with id 931e214f14f092bbf29413c57edba04c302573091ee3ae8d16f0f3ca01772169
Oct 30 00:45:40.043522 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:40.043488    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46" event={"ID":"343d735a-b95a-447f-869a-6cb18af8f934","Type":"ContainerStarted","Data":"931e214f14f092bbf29413c57edba04c302573091ee3ae8d16f0f3ca01772169"}
Oct 30 00:45:40.075496 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:40.075465    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"]
Oct 30 00:45:40.078715 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:40.078689    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e493142_9f44_43da_96b9_18d6408ed2d4.slice/crio-15ceafadddc4b3cff1d959bee2e90dce1d321f76f71155771181bca6ef2b12c8 WatchSource:0}: Error finding container 15ceafadddc4b3cff1d959bee2e90dce1d321f76f71155771181bca6ef2b12c8: Status 404 returned error can't find the container with id 15ceafadddc4b3cff1d959bee2e90dce1d321f76f71155771181bca6ef2b12c8
Oct 30 00:45:40.172104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:40.172027    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"]
Oct 30 00:45:40.176483 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:40.176447    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda33aa2d6_639a_4c47_8846_260b6b513e87.slice/crio-d98351bf53a4a62ab4024caf07fcd0aaf10da257f9a56082a7df0f4a4fdbe093 WatchSource:0}: Error finding container d98351bf53a4a62ab4024caf07fcd0aaf10da257f9a56082a7df0f4a4fdbe093: Status 404 returned error can't find the container with id d98351bf53a4a62ab4024caf07fcd0aaf10da257f9a56082a7df0f4a4fdbe093
Oct 30 00:45:40.270580 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:40.270411    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"]
Oct 30 00:45:40.273430 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:40.273393    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b857c3_dece_46a4_8f6d_6f800eb70e93.slice/crio-89119fb66ef7ad5cff5332db473dafaf8f287a9356729ae92a0785bb2dbae80f WatchSource:0}: Error finding container 89119fb66ef7ad5cff5332db473dafaf8f287a9356729ae92a0785bb2dbae80f: Status 404 returned error can't find the container with id 89119fb66ef7ad5cff5332db473dafaf8f287a9356729ae92a0785bb2dbae80f
Oct 30 00:45:40.428566 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:40.428486    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:40.538430 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:40.538396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:41.052800 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.052765    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" event={"ID":"a33aa2d6-639a-4c47-8846-260b6b513e87","Type":"ContainerStarted","Data":"d98351bf53a4a62ab4024caf07fcd0aaf10da257f9a56082a7df0f4a4fdbe093"}
Oct 30 00:45:41.054801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.054772    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" event={"ID":"63b857c3-dece-46a4-8f6d-6f800eb70e93","Type":"ContainerStarted","Data":"89119fb66ef7ad5cff5332db473dafaf8f287a9356729ae92a0785bb2dbae80f"}
Oct 30 00:45:41.057396 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.057369    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx" event={"ID":"1e493142-9f44-43da-96b9-18d6408ed2d4","Type":"ContainerStarted","Data":"15ceafadddc4b3cff1d959bee2e90dce1d321f76f71155771181bca6ef2b12c8"}
Oct 30 00:45:41.481647 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.481611    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"]
Oct 30 00:45:41.486403 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.486377    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.520441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.520373    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"]
Oct 30 00:45:41.530270 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.530209    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-wckrn\""
Oct 30 00:45:41.643728 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.643695    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"]
Oct 30 00:45:41.648578 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.647333    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.653306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.653025    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpkzp\" (UniqueName: \"kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.653306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.653104    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.653306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.653155    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.676622 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.676239    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"]
Oct 30 00:45:41.678335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.678099    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 00:45:41.678335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.678207    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 00:45:41.678335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.678297    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 00:45:41.678766 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.678732    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-spn7j\""
Oct 30 00:45:41.705795 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.705740    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:41.730219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.728787    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"]
Oct 30 00:45:41.733329 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.732721    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.751687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.751627    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"]
Oct 30 00:45:41.753910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.753643    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.753910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.753681    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.753910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.753713    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.753910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.753762    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.753910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.753788    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.753910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.753812    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.754608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754192    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.754732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754711    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.754810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754752    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpkzp\" (UniqueName: \"kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.754810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754795    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.754933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754828    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xppsj\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.754933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754896    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.754933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.754924    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.755543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.755525    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.756755 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.756679    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-qghb7\""
Oct 30 00:45:41.781860 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.780268    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"]
Oct 30 00:45:41.784735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.784198    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.799956 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.799889    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpkzp\" (UniqueName: \"kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.806526 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.806502    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"]
Oct 30 00:45:41.809271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.808402    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:45:41.839268 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.839236    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:41.853543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.853517    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-mdpxj\""
Oct 30 00:45:41.856095 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856071    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856168 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856119    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.856168 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856153    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856239 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856179    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856239 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856216    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzkp8\" (UniqueName: \"kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.856295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856263    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856293    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856320    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.856386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856344    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856394    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856446 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856433    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.856478 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.856464    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xppsj\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.857872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.857829    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.859493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.858447    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.859493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.858694    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.859493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.858963    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.859493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.859441    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.860458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.860429    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.861610 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.861564    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.886355 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.886281    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.887658 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.887607    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xppsj\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-ljrb4\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.953166 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.953060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958161    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958208    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958235    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958271    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w525l\" (UniqueName: \"kubernetes.io/projected/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-kube-api-access-w525l\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958313    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958374    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958400    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958434    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958487    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958517    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958542    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.958621    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzkp8\" (UniqueName: \"kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.959341    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.959732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.959655    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:41.978417 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.978134    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:41.993519 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:41.993418    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzkp8\" (UniqueName: \"kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:42.054903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.054623    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060696    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060734    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060806    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060836    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060887    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w525l\" (UniqueName: \"kubernetes.io/projected/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-kube-api-access-w525l\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060936    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.060995    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.061018    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.061050    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.061785    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.061900    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.062055    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.062283    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.062538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.062500    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.077337 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.077306    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.080817 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.080756    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.094898 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.094814    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w525l\" (UniqueName: \"kubernetes.io/projected/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-kube-api-access-w525l\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.107218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.105754    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec76aee0-2b4c-4ad0-b742-5bf48b05deea-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p\" (UID: \"ec76aee0-2b4c-4ad0-b742-5bf48b05deea\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.112751 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.112725    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:42.167246 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.167076    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"]
Oct 30 00:45:42.173193 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:42.173159    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58532f54_cf06_42fd_943b_e77be77a627e.slice/crio-4674c2e08a350e3ecf1b297f494807992aa99bde680aa13de108b0a8898701f8 WatchSource:0}: Error finding container 4674c2e08a350e3ecf1b297f494807992aa99bde680aa13de108b0a8898701f8: Status 404 returned error can't find the container with id 4674c2e08a350e3ecf1b297f494807992aa99bde680aa13de108b0a8898701f8
Oct 30 00:45:42.311541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.311515    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"]
Oct 30 00:45:42.317040 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:42.317012    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46950bd3_baa1_4289_bc9c_0a85b6548788.slice/crio-a3e9248f2354e2637d387a20ab87f2be1495998ededfd0c6e315e648e749295d WatchSource:0}: Error finding container a3e9248f2354e2637d387a20ab87f2be1495998ededfd0c6e315e648e749295d: Status 404 returned error can't find the container with id a3e9248f2354e2637d387a20ab87f2be1495998ededfd0c6e315e648e749295d
Oct 30 00:45:42.396983 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.396954    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"]
Oct 30 00:45:42.400257 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:42.400227    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a8d8f6_0759_449d_ad6f_1c3174066481.slice/crio-12be7e9b300408f7d71470a56044db26858a1fcd458570e8793a637e7aa5ed4b WatchSource:0}: Error finding container 12be7e9b300408f7d71470a56044db26858a1fcd458570e8793a637e7aa5ed4b: Status 404 returned error can't find the container with id 12be7e9b300408f7d71470a56044db26858a1fcd458570e8793a637e7aa5ed4b
Oct 30 00:45:42.455509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:42.455373    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"]
Oct 30 00:45:42.458884 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:45:42.458299    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec76aee0_2b4c_4ad0_b742_5bf48b05deea.slice/crio-6e63bc9de9a2f9d838c10890c9b81fcfb03a6169242e1adac49a3b7531faeacc WatchSource:0}: Error finding container 6e63bc9de9a2f9d838c10890c9b81fcfb03a6169242e1adac49a3b7531faeacc: Status 404 returned error can't find the container with id 6e63bc9de9a2f9d838c10890c9b81fcfb03a6169242e1adac49a3b7531faeacc
Oct 30 00:45:43.098441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:43.098282    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9" event={"ID":"58532f54-cf06-42fd-943b-e77be77a627e","Type":"ContainerStarted","Data":"4674c2e08a350e3ecf1b297f494807992aa99bde680aa13de108b0a8898701f8"}
Oct 30 00:45:43.123160 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:43.123117    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" event={"ID":"46950bd3-baa1-4289-bc9c-0a85b6548788","Type":"ContainerStarted","Data":"a3e9248f2354e2637d387a20ab87f2be1495998ededfd0c6e315e648e749295d"}
Oct 30 00:45:43.132454 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:43.132392    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" event={"ID":"11a8d8f6-0759-449d-ad6f-1c3174066481","Type":"ContainerStarted","Data":"12be7e9b300408f7d71470a56044db26858a1fcd458570e8793a637e7aa5ed4b"}
Oct 30 00:45:43.136312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:43.136250    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p" event={"ID":"ec76aee0-2b4c-4ad0-b742-5bf48b05deea","Type":"ContainerStarted","Data":"6e63bc9de9a2f9d838c10890c9b81fcfb03a6169242e1adac49a3b7531faeacc"}
Oct 30 00:45:44.981746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:44.981711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:45:50.226012 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:50.225218    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:45:50.226012 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:50.225311    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:45:50.226012 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:50.225357    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:45:50.380899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:50.380857    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:45:50.381020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:50.380969    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:45:50.381079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:50.381053    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:45:51.237914 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:51.237874    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p" event={"ID":"ec76aee0-2b4c-4ad0-b742-5bf48b05deea","Type":"ContainerStarted","Data":"2433d47e8af7a18894ab280f8dc1eaf054919fa02a3a75e85c8965e7fb07884c"}
Oct 30 00:45:51.242994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:51.242957    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" event={"ID":"46950bd3-baa1-4289-bc9c-0a85b6548788","Type":"ContainerStarted","Data":"b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"}
Oct 30 00:45:51.316915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:51.316819    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p" podStartSLOduration=2.553440456 podStartE2EDuration="10.316799345s" totalImagesPullingTime="7.763358889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:45:51.301603274 +0000 UTC m=+259.330913507" watchObservedRunningTime="2026-10-30 00:45:51.316799345 +0000 UTC m=+259.346109576"
Oct 30 00:45:51.350946 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:51.350895    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podStartSLOduration=2.291529004 podStartE2EDuration="10.350880368s" totalImagesPullingTime="8.059351364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:45:51.349602245 +0000 UTC m=+259.378912480" watchObservedRunningTime="2026-10-30 00:45:51.350880368 +0000 UTC m=+259.380190662"
Oct 30 00:45:51.979468 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:51.979432    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:51.992909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:51.986136    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:52.116728 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:52.115698    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:52.120300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:52.120106    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:52.254032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:52.253953    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:52.254512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:52.254321    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:52.255948 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:52.255624    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p"
Oct 30 00:45:52.255948 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:52.255893    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:45:53.267532 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:53.267499    4577 generic.go:356] "Generic (PLEG): container finished" podID="58532f54-cf06-42fd-943b-e77be77a627e" containerID="2fed60707d5fad9dcbb5695ccfe15c4a9adf7c88875d2313d26a77c75158124c" exitCode=0
Oct 30 00:45:53.267969 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:53.267943    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9" event={"ID":"58532f54-cf06-42fd-943b-e77be77a627e","Type":"ContainerDied","Data":"2fed60707d5fad9dcbb5695ccfe15c4a9adf7c88875d2313d26a77c75158124c"}
Oct 30 00:45:53.271084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:53.271057    4577 generic.go:356] "Generic (PLEG): container finished" podID="54acdd8a-5fe1-415d-b407-551e9937e971" containerID="b0a100b3ab2b355e3bb9d2b8d3131ad9998aa85b6c5311557bce7603b11bb5b1" exitCode=0
Oct 30 00:45:53.271175 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:53.271143    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl" event={"ID":"54acdd8a-5fe1-415d-b407-551e9937e971","Type":"ContainerDied","Data":"b0a100b3ab2b355e3bb9d2b8d3131ad9998aa85b6c5311557bce7603b11bb5b1"}
Oct 30 00:45:53.273183 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:53.273158    4577 generic.go:356] "Generic (PLEG): container finished" podID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerID="c8856f48316809d149c64558e1bb968137ecc4b196ee974682a55848d3e9dc81" exitCode=0
Oct 30 00:45:53.273491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:53.273465    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" event={"ID":"11a8d8f6-0759-449d-ad6f-1c3174066481","Type":"ContainerDied","Data":"c8856f48316809d149c64558e1bb968137ecc4b196ee974682a55848d3e9dc81"}
Oct 30 00:45:55.286958 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.286924    4577 generic.go:356] "Generic (PLEG): container finished" podID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerID="f273ac03604c421f51a69ed86639796a7d47d0096ddd08d864aa5cd78b5e66ba" exitCode=0
Oct 30 00:45:55.287483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.287059    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" event={"ID":"a33aa2d6-639a-4c47-8846-260b6b513e87","Type":"ContainerDied","Data":"f273ac03604c421f51a69ed86639796a7d47d0096ddd08d864aa5cd78b5e66ba"}
Oct 30 00:45:55.289317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.289287    4577 generic.go:356] "Generic (PLEG): container finished" podID="54acdd8a-5fe1-415d-b407-551e9937e971" containerID="7af72308b6a3ba63606a0e057cb69b3612c4ab45238a9a63809f070b2590f7d1" exitCode=0
Oct 30 00:45:55.289467 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.289333    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl" event={"ID":"54acdd8a-5fe1-415d-b407-551e9937e971","Type":"ContainerDied","Data":"7af72308b6a3ba63606a0e057cb69b3612c4ab45238a9a63809f070b2590f7d1"}
Oct 30 00:45:55.291362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.291341    4577 generic.go:356] "Generic (PLEG): container finished" podID="343d735a-b95a-447f-869a-6cb18af8f934" containerID="87be8d2e71cc122632c801cf84b20601b33eaf0b88a9ae9aa2464eeafa3c5472" exitCode=0
Oct 30 00:45:55.291609 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.291563    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46" event={"ID":"343d735a-b95a-447f-869a-6cb18af8f934","Type":"ContainerDied","Data":"87be8d2e71cc122632c801cf84b20601b33eaf0b88a9ae9aa2464eeafa3c5472"}
Oct 30 00:45:55.294392 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.294299    4577 generic.go:356] "Generic (PLEG): container finished" podID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerID="6aff35d57c4d9d04d5761a59c38f0fc1cbc5161b09c5d292fed51d6dc63a9a14" exitCode=0
Oct 30 00:45:55.294486 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.294391    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" event={"ID":"63b857c3-dece-46a4-8f6d-6f800eb70e93","Type":"ContainerDied","Data":"6aff35d57c4d9d04d5761a59c38f0fc1cbc5161b09c5d292fed51d6dc63a9a14"}
Oct 30 00:45:55.296753 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.296738    4577 generic.go:356] "Generic (PLEG): container finished" podID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerID="8f6af383d9e2f17eae7e99418dbeed124d9e6365974c4df3faf0241e9eed500f" exitCode=0
Oct 30 00:45:55.296826 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:55.296812    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx" event={"ID":"1e493142-9f44-43da-96b9-18d6408ed2d4","Type":"ContainerDied","Data":"8f6af383d9e2f17eae7e99418dbeed124d9e6365974c4df3faf0241e9eed500f"}
Oct 30 00:45:56.305335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.305256    4577 generic.go:356] "Generic (PLEG): container finished" podID="58532f54-cf06-42fd-943b-e77be77a627e" containerID="c8ecd65c3b345caa4c63036c5dc2db603c1f682c9557ea1d9c8ac80f920478b8" exitCode=0
Oct 30 00:45:56.305771 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.305376    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9" event={"ID":"58532f54-cf06-42fd-943b-e77be77a627e","Type":"ContainerDied","Data":"c8ecd65c3b345caa4c63036c5dc2db603c1f682c9557ea1d9c8ac80f920478b8"}
Oct 30 00:45:56.308256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.308142    4577 generic.go:356] "Generic (PLEG): container finished" podID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerID="c5c9e36a2368cf663510c4f091bf613ee75d50e5ea1c2e62b8658a40e3bd1613" exitCode=0
Oct 30 00:45:56.308256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.308232    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" event={"ID":"a33aa2d6-639a-4c47-8846-260b6b513e87","Type":"ContainerDied","Data":"c5c9e36a2368cf663510c4f091bf613ee75d50e5ea1c2e62b8658a40e3bd1613"}
Oct 30 00:45:56.311736 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.311712    4577 generic.go:356] "Generic (PLEG): container finished" podID="343d735a-b95a-447f-869a-6cb18af8f934" containerID="41d445bb606b342e7db82906c7a0ee13d35ada2aa929fa4add4b6fc9cd6a1833" exitCode=0
Oct 30 00:45:56.311833 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.311786    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46" event={"ID":"343d735a-b95a-447f-869a-6cb18af8f934","Type":"ContainerDied","Data":"41d445bb606b342e7db82906c7a0ee13d35ada2aa929fa4add4b6fc9cd6a1833"}
Oct 30 00:45:56.315214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.315122    4577 generic.go:356] "Generic (PLEG): container finished" podID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerID="2c84c95122eb190bac11ab3553ccb2dd52cf4b662ce5feace5c66f0a8fcf13ea" exitCode=0
Oct 30 00:45:56.315214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.315153    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" event={"ID":"63b857c3-dece-46a4-8f6d-6f800eb70e93","Type":"ContainerDied","Data":"2c84c95122eb190bac11ab3553ccb2dd52cf4b662ce5feace5c66f0a8fcf13ea"}
Oct 30 00:45:56.317368 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.317346    4577 generic.go:356] "Generic (PLEG): container finished" podID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerID="c784ff9f441e1b886e6dac3b2b7b0e77c420938ae445ad1c7acf9010750901ba" exitCode=0
Oct 30 00:45:56.317487 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:56.317450    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx" event={"ID":"1e493142-9f44-43da-96b9-18d6408ed2d4","Type":"ContainerDied","Data":"c784ff9f441e1b886e6dac3b2b7b0e77c420938ae445ad1c7acf9010750901ba"}
Oct 30 00:45:57.028031 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:45:57.027910    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:45:57.358780 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:57.358626    4577 generic.go:356] "Generic (PLEG): container finished" podID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerID="845c0d58213925dd622222c939b3a583e44de5df6e4c4a1add5648dae4df852e" exitCode=0
Oct 30 00:45:57.358780 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:45:57.358696    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" event={"ID":"11a8d8f6-0759-449d-ad6f-1c3174066481","Type":"ContainerDied","Data":"845c0d58213925dd622222c939b3a583e44de5df6e4c4a1add5648dae4df852e"}
Oct 30 00:46:10.507175 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:10.507133    4577 generic.go:356] "Generic (PLEG): container finished" podID="54acdd8a-5fe1-415d-b407-551e9937e971" containerID="3154d046bc232d9dd1f8b02b9e72fd2258dac7c6291d6a3118c74ca87939daf9" exitCode=0
Oct 30 00:46:10.507619 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:10.507220    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl" event={"ID":"54acdd8a-5fe1-415d-b407-551e9937e971","Type":"ContainerDied","Data":"3154d046bc232d9dd1f8b02b9e72fd2258dac7c6291d6a3118c74ca87939daf9"}
Oct 30 00:46:11.514698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.514668    4577 generic.go:356] "Generic (PLEG): container finished" podID="58532f54-cf06-42fd-943b-e77be77a627e" containerID="ca811578fda185eb7a380cb60aa737cfad51131d475f1242ecda30ea7b36ee24" exitCode=0
Oct 30 00:46:11.515090 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.514753    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9" event={"ID":"58532f54-cf06-42fd-943b-e77be77a627e","Type":"ContainerDied","Data":"ca811578fda185eb7a380cb60aa737cfad51131d475f1242ecda30ea7b36ee24"}
Oct 30 00:46:11.517246 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.517222    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" event={"ID":"a33aa2d6-639a-4c47-8846-260b6b513e87","Type":"ContainerStarted","Data":"5d60fd81e56c94c647b0d87676594974eb778f83392594ac3de657a9dabdc5ba"}
Oct 30 00:46:11.519475 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.519450    4577 generic.go:356] "Generic (PLEG): container finished" podID="343d735a-b95a-447f-869a-6cb18af8f934" containerID="be9f974d43b8343384a2ea50658e233642ef6649b7868e503879ada6e40da441" exitCode=0
Oct 30 00:46:11.519475 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.519472    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46" event={"ID":"343d735a-b95a-447f-869a-6cb18af8f934","Type":"ContainerDied","Data":"be9f974d43b8343384a2ea50658e233642ef6649b7868e503879ada6e40da441"}
Oct 30 00:46:11.521478 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.521457    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" event={"ID":"63b857c3-dece-46a4-8f6d-6f800eb70e93","Type":"ContainerStarted","Data":"c9b0afcb24e484ad2940720d9e0d33abb55830a16f9e96109741037b47ec873a"}
Oct 30 00:46:11.523317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.523274    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" event={"ID":"11a8d8f6-0759-449d-ad6f-1c3174066481","Type":"ContainerStarted","Data":"14513a0f660b40f13933bed4747481c654de653736b50c42f742cf18b4504191"}
Oct 30 00:46:11.525298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.525280    4577 generic.go:356] "Generic (PLEG): container finished" podID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerID="a51fc3e1d10d45fd3bbed6c474f9ad0eab080afdf33879b0a8ded478aba2d790" exitCode=0
Oct 30 00:46:11.525400 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.525347    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx" event={"ID":"1e493142-9f44-43da-96b9-18d6408ed2d4","Type":"ContainerDied","Data":"a51fc3e1d10d45fd3bbed6c474f9ad0eab080afdf33879b0a8ded478aba2d790"}
Oct 30 00:46:11.610703 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.610642    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" podStartSLOduration=2.107787826 podStartE2EDuration="32.610628204s" totalImagesPullingTime="30.502840378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:45:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:11.609390549 +0000 UTC m=+279.638700772" watchObservedRunningTime="2026-10-30 00:46:11.610628204 +0000 UTC m=+279.639938434"
Oct 30 00:46:11.640411 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.640387    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:46:11.702960 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.702916    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" podStartSLOduration=2.979318459 podStartE2EDuration="30.70290083s" totalImagesPullingTime="27.723582371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:45:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:11.702551932 +0000 UTC m=+279.731862175" watchObservedRunningTime="2026-10-30 00:46:11.70290083 +0000 UTC m=+279.732211060"
Oct 30 00:46:11.772407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.772375    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle\") pod \"54acdd8a-5fe1-415d-b407-551e9937e971\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") "
Oct 30 00:46:11.772541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.772436    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj\" (UniqueName: \"kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj\") pod \"54acdd8a-5fe1-415d-b407-551e9937e971\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") "
Oct 30 00:46:11.772541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.772461    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util\") pod \"54acdd8a-5fe1-415d-b407-551e9937e971\" (UID: \"54acdd8a-5fe1-415d-b407-551e9937e971\") "
Oct 30 00:46:11.773017 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.772992    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle" pod "54acdd8a-5fe1-415d-b407-551e9937e971" (UID: "54acdd8a-5fe1-415d-b407-551e9937e971"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:11.774772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.774748    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj" pod "54acdd8a-5fe1-415d-b407-551e9937e971" (UID: "54acdd8a-5fe1-415d-b407-551e9937e971"). InnerVolumeSpecName "kube-api-access-pmctj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:11.777788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.777767    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util" pod "54acdd8a-5fe1-415d-b407-551e9937e971" (UID: "54acdd8a-5fe1-415d-b407-551e9937e971"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:11.795680 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.795633    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" podStartSLOduration=2.593452192 podStartE2EDuration="32.795619874s" totalImagesPullingTime="30.202167682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:45:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:11.795560359 +0000 UTC m=+279.824870588" watchObservedRunningTime="2026-10-30 00:46:11.795619874 +0000 UTC m=+279.824930103"
Oct 30 00:46:11.873601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.873581    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:11.873601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.873602    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmctj\" (UniqueName: \"kubernetes.io/projected/54acdd8a-5fe1-415d-b407-551e9937e971-kube-api-access-pmctj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:11.873730 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:11.873611    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54acdd8a-5fe1-415d-b407-551e9937e971-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.531003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.530969    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl" event={"ID":"54acdd8a-5fe1-415d-b407-551e9937e971","Type":"ContainerDied","Data":"62ac07892eb1936fe658465ec38871afd6e54c6fc09bd083dae61bb637798c61"}
Oct 30 00:46:12.531407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.531018    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62ac07892eb1936fe658465ec38871afd6e54c6fc09bd083dae61bb637798c61"
Oct 30 00:46:12.531407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.530980    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71xjqnl"
Oct 30 00:46:12.533256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.533226    4577 generic.go:356] "Generic (PLEG): container finished" podID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerID="c9b0afcb24e484ad2940720d9e0d33abb55830a16f9e96109741037b47ec873a" exitCode=0
Oct 30 00:46:12.533378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.533304    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" event={"ID":"63b857c3-dece-46a4-8f6d-6f800eb70e93","Type":"ContainerDied","Data":"c9b0afcb24e484ad2940720d9e0d33abb55830a16f9e96109741037b47ec873a"}
Oct 30 00:46:12.535413 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.535395    4577 generic.go:356] "Generic (PLEG): container finished" podID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerID="14513a0f660b40f13933bed4747481c654de653736b50c42f742cf18b4504191" exitCode=0
Oct 30 00:46:12.535500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.535479    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" event={"ID":"11a8d8f6-0759-449d-ad6f-1c3174066481","Type":"ContainerDied","Data":"14513a0f660b40f13933bed4747481c654de653736b50c42f742cf18b4504191"}
Oct 30 00:46:12.537448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.537408    4577 generic.go:356] "Generic (PLEG): container finished" podID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerID="5d60fd81e56c94c647b0d87676594974eb778f83392594ac3de657a9dabdc5ba" exitCode=0
Oct 30 00:46:12.537524 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.537502    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" event={"ID":"a33aa2d6-639a-4c47-8846-260b6b513e87","Type":"ContainerDied","Data":"5d60fd81e56c94c647b0d87676594974eb778f83392594ac3de657a9dabdc5ba"}
Oct 30 00:46:12.666787 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.666762    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:46:12.723445 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.723426    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:46:12.725757 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.725738    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:46:12.782108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782034    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util\") pod \"343d735a-b95a-447f-869a-6cb18af8f934\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") "
Oct 30 00:46:12.782265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782106    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle\") pod \"58532f54-cf06-42fd-943b-e77be77a627e\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") "
Oct 30 00:46:12.782265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782137    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp\" (UniqueName: \"kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp\") pod \"58532f54-cf06-42fd-943b-e77be77a627e\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") "
Oct 30 00:46:12.782265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782163    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle\") pod \"343d735a-b95a-447f-869a-6cb18af8f934\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") "
Oct 30 00:46:12.782265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782202    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7\" (UniqueName: \"kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7\") pod \"343d735a-b95a-447f-869a-6cb18af8f934\" (UID: \"343d735a-b95a-447f-869a-6cb18af8f934\") "
Oct 30 00:46:12.782265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782227    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util\") pod \"58532f54-cf06-42fd-943b-e77be77a627e\" (UID: \"58532f54-cf06-42fd-943b-e77be77a627e\") "
Oct 30 00:46:12.782512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782281    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh\" (UniqueName: \"kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh\") pod \"1e493142-9f44-43da-96b9-18d6408ed2d4\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") "
Oct 30 00:46:12.782512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782322    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util\") pod \"1e493142-9f44-43da-96b9-18d6408ed2d4\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") "
Oct 30 00:46:12.782512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782352    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle\") pod \"1e493142-9f44-43da-96b9-18d6408ed2d4\" (UID: \"1e493142-9f44-43da-96b9-18d6408ed2d4\") "
Oct 30 00:46:12.782657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782512    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle" pod "58532f54-cf06-42fd-943b-e77be77a627e" (UID: "58532f54-cf06-42fd-943b-e77be77a627e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:12.782709 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.782666    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.783150 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.783125    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle" pod "1e493142-9f44-43da-96b9-18d6408ed2d4" (UID: "1e493142-9f44-43da-96b9-18d6408ed2d4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:12.784068 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.784039    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle" pod "343d735a-b95a-447f-869a-6cb18af8f934" (UID: "343d735a-b95a-447f-869a-6cb18af8f934"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:12.786394 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.784901    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp" pod "58532f54-cf06-42fd-943b-e77be77a627e" (UID: "58532f54-cf06-42fd-943b-e77be77a627e"). InnerVolumeSpecName "kube-api-access-tpkzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:12.786394 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.785638    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7" pod "343d735a-b95a-447f-869a-6cb18af8f934" (UID: "343d735a-b95a-447f-869a-6cb18af8f934"). InnerVolumeSpecName "kube-api-access-bbkv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:12.787762 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.787731    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh" pod "1e493142-9f44-43da-96b9-18d6408ed2d4" (UID: "1e493142-9f44-43da-96b9-18d6408ed2d4"). InnerVolumeSpecName "kube-api-access-nzqwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:12.788357 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.788332    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util" pod "343d735a-b95a-447f-869a-6cb18af8f934" (UID: "343d735a-b95a-447f-869a-6cb18af8f934"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:12.789678 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.789656    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util" pod "1e493142-9f44-43da-96b9-18d6408ed2d4" (UID: "1e493142-9f44-43da-96b9-18d6408ed2d4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:12.790196 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.790170    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util" pod "58532f54-cf06-42fd-943b-e77be77a627e" (UID: "58532f54-cf06-42fd-943b-e77be77a627e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:12.884090 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884064    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzqwh\" (UniqueName: \"kubernetes.io/projected/1e493142-9f44-43da-96b9-18d6408ed2d4-kube-api-access-nzqwh\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884090 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884086    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884232 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884094    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e493142-9f44-43da-96b9-18d6408ed2d4-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884232 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884105    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884232 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884113    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpkzp\" (UniqueName: \"kubernetes.io/projected/58532f54-cf06-42fd-943b-e77be77a627e-kube-api-access-tpkzp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884232 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884121    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/343d735a-b95a-447f-869a-6cb18af8f934-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884232 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884129    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbkv7\" (UniqueName: \"kubernetes.io/projected/343d735a-b95a-447f-869a-6cb18af8f934-kube-api-access-bbkv7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:12.884232 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:12.884137    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58532f54-cf06-42fd-943b-e77be77a627e-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.543465 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.543432    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46"
Oct 30 00:46:13.543925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.543426    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6n6k46" event={"ID":"343d735a-b95a-447f-869a-6cb18af8f934","Type":"ContainerDied","Data":"931e214f14f092bbf29413c57edba04c302573091ee3ae8d16f0f3ca01772169"}
Oct 30 00:46:13.543925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.543555    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="931e214f14f092bbf29413c57edba04c302573091ee3ae8d16f0f3ca01772169"
Oct 30 00:46:13.545293 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.545271    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx"
Oct 30 00:46:13.545418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.545301    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7k95nx" event={"ID":"1e493142-9f44-43da-96b9-18d6408ed2d4","Type":"ContainerDied","Data":"15ceafadddc4b3cff1d959bee2e90dce1d321f76f71155771181bca6ef2b12c8"}
Oct 30 00:46:13.545418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.545345    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15ceafadddc4b3cff1d959bee2e90dce1d321f76f71155771181bca6ef2b12c8"
Oct 30 00:46:13.547278 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.547262    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9"
Oct 30 00:46:13.547377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.547309    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vhvc9" event={"ID":"58532f54-cf06-42fd-943b-e77be77a627e","Type":"ContainerDied","Data":"4674c2e08a350e3ecf1b297f494807992aa99bde680aa13de108b0a8898701f8"}
Oct 30 00:46:13.547377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.547355    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4674c2e08a350e3ecf1b297f494807992aa99bde680aa13de108b0a8898701f8"
Oct 30 00:46:13.685073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.685047    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:46:13.718522 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.718504    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:46:13.720752 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.720736    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:46:13.792238 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792209    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle\") pod \"63b857c3-dece-46a4-8f6d-6f800eb70e93\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") "
Oct 30 00:46:13.792238 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792245    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle\") pod \"11a8d8f6-0759-449d-ad6f-1c3174066481\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") "
Oct 30 00:46:13.792491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792284    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7\" (UniqueName: \"kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7\") pod \"63b857c3-dece-46a4-8f6d-6f800eb70e93\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") "
Oct 30 00:46:13.792491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792301    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8\" (UniqueName: \"kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8\") pod \"11a8d8f6-0759-449d-ad6f-1c3174066481\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") "
Oct 30 00:46:13.792491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792325    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p\" (UniqueName: \"kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p\") pod \"a33aa2d6-639a-4c47-8846-260b6b513e87\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") "
Oct 30 00:46:13.792491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792341    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util\") pod \"11a8d8f6-0759-449d-ad6f-1c3174066481\" (UID: \"11a8d8f6-0759-449d-ad6f-1c3174066481\") "
Oct 30 00:46:13.792491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792407    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle\") pod \"a33aa2d6-639a-4c47-8846-260b6b513e87\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") "
Oct 30 00:46:13.792730 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.792502    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util\") pod \"a33aa2d6-639a-4c47-8846-260b6b513e87\" (UID: \"a33aa2d6-639a-4c47-8846-260b6b513e87\") "
Oct 30 00:46:13.793190 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.793158    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle" pod "63b857c3-dece-46a4-8f6d-6f800eb70e93" (UID: "63b857c3-dece-46a4-8f6d-6f800eb70e93"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:13.793284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.793165    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle" pod "11a8d8f6-0759-449d-ad6f-1c3174066481" (UID: "11a8d8f6-0759-449d-ad6f-1c3174066481"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:13.793284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.793227    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle" pod "a33aa2d6-639a-4c47-8846-260b6b513e87" (UID: "a33aa2d6-639a-4c47-8846-260b6b513e87"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:13.794753 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.794684    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util\") pod \"63b857c3-dece-46a4-8f6d-6f800eb70e93\" (UID: \"63b857c3-dece-46a4-8f6d-6f800eb70e93\") "
Oct 30 00:46:13.795083 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795056    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p" pod "a33aa2d6-639a-4c47-8846-260b6b513e87" (UID: "a33aa2d6-639a-4c47-8846-260b6b513e87"). InnerVolumeSpecName "kube-api-access-wlt9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:13.795189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795098    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7" pod "63b857c3-dece-46a4-8f6d-6f800eb70e93" (UID: "63b857c3-dece-46a4-8f6d-6f800eb70e93"). InnerVolumeSpecName "kube-api-access-ppqd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:13.795189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795162    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlt9p\" (UniqueName: \"kubernetes.io/projected/a33aa2d6-639a-4c47-8846-260b6b513e87-kube-api-access-wlt9p\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.795189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795179    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.795342 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795191    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.795342 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795230    4577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-bundle\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.795474 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.795458    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8" pod "11a8d8f6-0759-449d-ad6f-1c3174066481" (UID: "11a8d8f6-0759-449d-ad6f-1c3174066481"). InnerVolumeSpecName "kube-api-access-gzkp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:13.798568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.798545    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util" pod "a33aa2d6-639a-4c47-8846-260b6b513e87" (UID: "a33aa2d6-639a-4c47-8846-260b6b513e87"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:13.798776 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.798759    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util" pod "11a8d8f6-0759-449d-ad6f-1c3174066481" (UID: "11a8d8f6-0759-449d-ad6f-1c3174066481"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:13.800280 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.800259    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util" pod "63b857c3-dece-46a4-8f6d-6f800eb70e93" (UID: "63b857c3-dece-46a4-8f6d-6f800eb70e93"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:13.895822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.895794    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a33aa2d6-639a-4c47-8846-260b6b513e87-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.895822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.895817    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63b857c3-dece-46a4-8f6d-6f800eb70e93-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.895822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.895826    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppqd7\" (UniqueName: \"kubernetes.io/projected/63b857c3-dece-46a4-8f6d-6f800eb70e93-kube-api-access-ppqd7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.896031 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.895836    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzkp8\" (UniqueName: \"kubernetes.io/projected/11a8d8f6-0759-449d-ad6f-1c3174066481-kube-api-access-gzkp8\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:13.896031 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:13.895868    4577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11a8d8f6-0759-449d-ad6f-1c3174066481-util\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:14.553528 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.553480    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh" event={"ID":"a33aa2d6-639a-4c47-8846-260b6b513e87","Type":"ContainerDied","Data":"d98351bf53a4a62ab4024caf07fcd0aaf10da257f9a56082a7df0f4a4fdbe093"}
Oct 30 00:46:14.554003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.553548    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d98351bf53a4a62ab4024caf07fcd0aaf10da257f9a56082a7df0f4a4fdbe093"
Oct 30 00:46:14.554003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.553503    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cc8cxh"
Oct 30 00:46:14.556049 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.555439    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8" event={"ID":"63b857c3-dece-46a4-8f6d-6f800eb70e93","Type":"ContainerDied","Data":"89119fb66ef7ad5cff5332db473dafaf8f287a9356729ae92a0785bb2dbae80f"}
Oct 30 00:46:14.556049 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.555449    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agtjw8"
Oct 30 00:46:14.556049 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.555481    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89119fb66ef7ad5cff5332db473dafaf8f287a9356729ae92a0785bb2dbae80f"
Oct 30 00:46:14.557560 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.557538    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf" event={"ID":"11a8d8f6-0759-449d-ad6f-1c3174066481","Type":"ContainerDied","Data":"12be7e9b300408f7d71470a56044db26858a1fcd458570e8793a637e7aa5ed4b"}
Oct 30 00:46:14.557661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.557578    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12be7e9b300408f7d71470a56044db26858a1fcd458570e8793a637e7aa5ed4b"
Oct 30 00:46:14.557661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:14.557583    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cn6ppf"
Oct 30 00:46:16.250994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:16.250961    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:16.361770 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:16.361738    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:17.196872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.196828    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:17.339458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339427    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"]
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339750    4577 cpu_manager.go:414] "containerMap: removing container" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="extract"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339763    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="extract"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339774    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="pull"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339779    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="pull"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339787    4577 cpu_manager.go:414] "containerMap: removing container" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="pull"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339792    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="pull"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339805    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="pull"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339812    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="pull"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339818    4577 cpu_manager.go:414] "containerMap: removing container" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="extract"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339822    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="extract"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339830    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="util"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339834    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="util"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339862    4577 cpu_manager.go:414] "containerMap: removing container" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="util"
Oct 30 00:46:17.339868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339868    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339877    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339887    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339895    4577 cpu_manager.go:414] "containerMap: removing container" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339900    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339908    4577 cpu_manager.go:414] "containerMap: removing container" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339912    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339917    4577 cpu_manager.go:414] "containerMap: removing container" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339921    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339927    4577 cpu_manager.go:414] "containerMap: removing container" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339931    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339936    4577 cpu_manager.go:414] "containerMap: removing container" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339940    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339948    4577 cpu_manager.go:414] "containerMap: removing container" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339951    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339962    4577 cpu_manager.go:414] "containerMap: removing container" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339969    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339975    4577 cpu_manager.go:414] "containerMap: removing container" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339979    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="pull"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339986    4577 cpu_manager.go:414] "containerMap: removing container" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339990    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339996    4577 cpu_manager.go:414] "containerMap: removing container" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.339999    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="util"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340005    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340009    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340020    4577 cpu_manager.go:414] "containerMap: removing container" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340024    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340029    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340033    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340085    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a33aa2d6-639a-4c47-8846-260b6b513e87" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340094    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="343d735a-b95a-447f-869a-6cb18af8f934" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340099    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e493142-9f44-43da-96b9-18d6408ed2d4" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340105    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="63b857c3-dece-46a4-8f6d-6f800eb70e93" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340111    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="54acdd8a-5fe1-415d-b407-551e9937e971" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340118    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="58532f54-cf06-42fd-943b-e77be77a627e" containerName="extract"
Oct 30 00:46:17.340336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.340129    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="11a8d8f6-0759-449d-ad6f-1c3174066481" containerName="extract"
Oct 30 00:46:17.367837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.367812    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.401748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.401715    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 00:46:17.401959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.401839    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-lkrjc\""
Oct 30 00:46:17.401959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.401873    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 00:46:17.401959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.401894    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 00:46:17.401959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.401896    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 00:46:17.402194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.402014    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 00:46:17.403639 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.403614    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"]
Oct 30 00:46:17.427386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.427361    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0021c2fc-262f-4746-97c7-ffdd76048d72-tmp\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.427500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.427394    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0021c2fc-262f-4746-97c7-ffdd76048d72-manifests\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.427500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.427412    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rsw4\" (UniqueName: \"kubernetes.io/projected/0021c2fc-262f-4746-97c7-ffdd76048d72-kube-api-access-9rsw4\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.427567 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.427514    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.427567 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.427551    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.427628 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.427572    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.528628 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.528594    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.528829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.528633    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.528829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.528770    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0021c2fc-262f-4746-97c7-ffdd76048d72-tmp\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.528829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.528809    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0021c2fc-262f-4746-97c7-ffdd76048d72-manifests\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.529009 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.528922    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rsw4\" (UniqueName: \"kubernetes.io/projected/0021c2fc-262f-4746-97c7-ffdd76048d72-kube-api-access-9rsw4\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.529009 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.528992    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.529262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.529234    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0021c2fc-262f-4746-97c7-ffdd76048d72-manifests\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.531337 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.531313    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.531337 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.531330    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.531591 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.531573    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0021c2fc-262f-4746-97c7-ffdd76048d72-tmp\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.531695 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.531682    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0021c2fc-262f-4746-97c7-ffdd76048d72-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.551863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.551589    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:17.558221 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.558188    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rsw4\" (UniqueName: \"kubernetes.io/projected/0021c2fc-262f-4746-97c7-ffdd76048d72-kube-api-access-9rsw4\") pod \"opendatahub-operator-controller-manager-f7f9fcc75-p7hlb\" (UID: \"0021c2fc-262f-4746-97c7-ffdd76048d72\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.665473 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.665402    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:17.680926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.680894    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:17.786611 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:17.786519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:18.038277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:18.038099    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"]
Oct 30 00:46:18.041065 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:18.041040    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0021c2fc_262f_4746_97c7_ffdd76048d72.slice/crio-e229bc5315295c4d394310ca44c62a7998c55518eac49f5aced252998476d771 WatchSource:0}: Error finding container e229bc5315295c4d394310ca44c62a7998c55518eac49f5aced252998476d771: Status 404 returned error can't find the container with id e229bc5315295c4d394310ca44c62a7998c55518eac49f5aced252998476d771
Oct 30 00:46:18.070456 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:18.070429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:18.577385 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:18.577349    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb" event={"ID":"0021c2fc-262f-4746-97c7-ffdd76048d72","Type":"ContainerStarted","Data":"e229bc5315295c4d394310ca44c62a7998c55518eac49f5aced252998476d771"}
Oct 30 00:46:18.934167 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:18.934082    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:19.644796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:19.644766    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:20.646703 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:20.646664    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:20.867999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:20.867973    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:21.002592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:21.002563    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:21.595505 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:21.595421    4577 generic.go:356] "Generic (PLEG): container finished" podID="0021c2fc-262f-4746-97c7-ffdd76048d72" containerID="6b1b78c0a1afbacc81437598f45207d055df04e727ed8961c6b8cbe2b4cbff98" exitCode=0
Oct 30 00:46:21.595641 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:21.595500    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb" event={"ID":"0021c2fc-262f-4746-97c7-ffdd76048d72","Type":"ContainerDied","Data":"6b1b78c0a1afbacc81437598f45207d055df04e727ed8961c6b8cbe2b4cbff98"}
Oct 30 00:46:21.834673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:21.834645    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:22.601633 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:22.601592    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb" event={"ID":"0021c2fc-262f-4746-97c7-ffdd76048d72","Type":"ContainerStarted","Data":"7956e9b5c984e2093a0564feed207165e45c83d6a683f69d7e3f5a799ca21029"}
Oct 30 00:46:22.601822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:22.601736    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:22.706836 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:22.706784    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb" podStartSLOduration=2.180313549 podStartE2EDuration="5.706770028s" totalImagesPullingTime="3.116000888s" totalInitContainerRuntime="410.455591ms" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:22.691767544 +0000 UTC m=+290.721077774" watchObservedRunningTime="2026-10-30 00:46:22.706770028 +0000 UTC m=+290.736080258"
Oct 30 00:46:25.075656 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:25.075624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:25.185875 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:25.185823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:27.012242 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:46:27.012198    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:46:27.220135 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.220103    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"]
Oct 30 00:46:27.226412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.226393    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.237555 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.237528    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"]
Oct 30 00:46:27.243088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.243065    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 00:46:27.249671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.249647    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 00:46:27.316045 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.315973    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mfxs4\""
Oct 30 00:46:27.316271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.316252    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-5dcf6\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.316311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.316290    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96ntv\" (UniqueName: \"kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv\") pod \"kuadrant-operator-controller-manager-cf8469d7b-5dcf6\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.417233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.417207    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96ntv\" (UniqueName: \"kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv\") pod \"kuadrant-operator-controller-manager-cf8469d7b-5dcf6\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.417380 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.417277    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-5dcf6\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.417648 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.417631    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-5dcf6\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.450763 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.450741    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96ntv\" (UniqueName: \"kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv\") pod \"kuadrant-operator-controller-manager-cf8469d7b-5dcf6\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.539569 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.539540    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:27.862327 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:27.862295    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"]
Oct 30 00:46:27.868178 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:27.868149    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f62fb6c_f30e_4b2a_9a50_26aa470ef5aa.slice/crio-29e7b28c116053461bde15b19be6bbdff92cd52f30a913d3382d24e515500c0a WatchSource:0}: Error finding container 29e7b28c116053461bde15b19be6bbdff92cd52f30a913d3382d24e515500c0a: Status 404 returned error can't find the container with id 29e7b28c116053461bde15b19be6bbdff92cd52f30a913d3382d24e515500c0a
Oct 30 00:46:28.630280 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:28.630243    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6" event={"ID":"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa","Type":"ContainerStarted","Data":"29e7b28c116053461bde15b19be6bbdff92cd52f30a913d3382d24e515500c0a"}
Oct 30 00:46:28.674121 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:28.674086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:30.454909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:30.454877    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:31.441590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.441553    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"]
Oct 30 00:46:31.445373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.445347    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:31.464871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.464827    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"]
Oct 30 00:46:31.477271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.477190    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qg9b6\""
Oct 30 00:46:31.477671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.477565    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 00:46:31.558925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.558794    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rj2p\" (UniqueName: \"kubernetes.io/projected/e49ab448-f19b-475e-add1-ad820ccc0eaa-kube-api-access-5rj2p\") pod \"dns-operator-controller-manager-8557cbdb84-nrc8l\" (UID: \"e49ab448-f19b-475e-add1-ad820ccc0eaa\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:31.659710 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.659680    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rj2p\" (UniqueName: \"kubernetes.io/projected/e49ab448-f19b-475e-add1-ad820ccc0eaa-kube-api-access-5rj2p\") pod \"dns-operator-controller-manager-8557cbdb84-nrc8l\" (UID: \"e49ab448-f19b-475e-add1-ad820ccc0eaa\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:31.685136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.685104    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rj2p\" (UniqueName: \"kubernetes.io/projected/e49ab448-f19b-475e-add1-ad820ccc0eaa-kube-api-access-5rj2p\") pod \"dns-operator-controller-manager-8557cbdb84-nrc8l\" (UID: \"e49ab448-f19b-475e-add1-ad820ccc0eaa\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:31.760574 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.760555    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:31.847297 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:31.847268    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:32.097294 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:32.097273    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"]
Oct 30 00:46:32.651120 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:32.651087    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6" event={"ID":"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa","Type":"ContainerStarted","Data":"da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789"}
Oct 30 00:46:32.651584 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:32.651214    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:32.652444 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:32.652423    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l" event={"ID":"e49ab448-f19b-475e-add1-ad820ccc0eaa","Type":"ContainerStarted","Data":"867637fa6ed46b42beeff0486096671bb958c2a1e627035cd37b4518de17f17b"}
Oct 30 00:46:32.693562 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:32.693529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:32.711579 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:32.711537    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6" podStartSLOduration=1.879129295 podStartE2EDuration="5.711525097s" totalImagesPullingTime="3.832395802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:32.710028231 +0000 UTC m=+300.739338463" watchObservedRunningTime="2026-10-30 00:46:32.711525097 +0000 UTC m=+300.740835325"
Oct 30 00:46:33.285276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:33.285244    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:33.608747 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:33.608678    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-f7f9fcc75-p7hlb"
Oct 30 00:46:34.142246 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.142223    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 00:46:34.149204 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.149183    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"]
Oct 30 00:46:34.152948 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.152930    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.177309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.177253    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 00:46:34.177309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.177302    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:46:34.180011 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.177767    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9xw96\""
Oct 30 00:46:34.180011 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.179806    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"]
Oct 30 00:46:34.284753 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.284725    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnqwf\" (UniqueName: \"kubernetes.io/projected/3efff35d-4ec4-494f-ba03-9264a12e8639-kube-api-access-cnqwf\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc\" (UID: \"3efff35d-4ec4-494f-ba03-9264a12e8639\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.285030 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.285004    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3efff35d-4ec4-494f-ba03-9264a12e8639-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc\" (UID: \"3efff35d-4ec4-494f-ba03-9264a12e8639\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.386236 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.386153    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3efff35d-4ec4-494f-ba03-9264a12e8639-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc\" (UID: \"3efff35d-4ec4-494f-ba03-9264a12e8639\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.386399 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.386248    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqwf\" (UniqueName: \"kubernetes.io/projected/3efff35d-4ec4-494f-ba03-9264a12e8639-kube-api-access-cnqwf\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc\" (UID: \"3efff35d-4ec4-494f-ba03-9264a12e8639\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.386535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.386513    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3efff35d-4ec4-494f-ba03-9264a12e8639-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc\" (UID: \"3efff35d-4ec4-494f-ba03-9264a12e8639\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.419085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.419057    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnqwf\" (UniqueName: \"kubernetes.io/projected/3efff35d-4ec4-494f-ba03-9264a12e8639-kube-api-access-cnqwf\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc\" (UID: \"3efff35d-4ec4-494f-ba03-9264a12e8639\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.488276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.488244    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"
Oct 30 00:46:34.668935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.668814    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l" event={"ID":"e49ab448-f19b-475e-add1-ad820ccc0eaa","Type":"ContainerStarted","Data":"3bfee966b69dfad63ee91d2ef4dd352c1f34a347a01fe4e2467dc2b2abf2a7d3"}
Oct 30 00:46:34.669187 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.669158    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:34.725368 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.725314    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l" podStartSLOduration=1.701182013 podStartE2EDuration="3.7252984s" totalImagesPullingTime="2.024116387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:34.724422798 +0000 UTC m=+302.753733046" watchObservedRunningTime="2026-10-30 00:46:34.7252984 +0000 UTC m=+302.754608630"
Oct 30 00:46:34.729243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.729220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:34.820365 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.820343    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc"]
Oct 30 00:46:34.823150 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:34.823124    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3efff35d_4ec4_494f_ba03_9264a12e8639.slice/crio-83219ffbb190ede40d8f01b0984e8a7a8326ede5b45f25bc5719b79d6ce37199 WatchSource:0}: Error finding container 83219ffbb190ede40d8f01b0984e8a7a8326ede5b45f25bc5719b79d6ce37199: Status 404 returned error can't find the container with id 83219ffbb190ede40d8f01b0984e8a7a8326ede5b45f25bc5719b79d6ce37199
Oct 30 00:46:34.825693 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.825677    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:46:34.964081 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:34.963965    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:35.086411 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:35.086380    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:35.675356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:35.675311    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc" event={"ID":"3efff35d-4ec4-494f-ba03-9264a12e8639","Type":"ContainerStarted","Data":"83219ffbb190ede40d8f01b0984e8a7a8326ede5b45f25bc5719b79d6ce37199"}
Oct 30 00:46:36.311269 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.311240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:36.501791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.501762    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:36.643541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.643465    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"]
Oct 30 00:46:36.646952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.646936    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:36.681909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.681876    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc" event={"ID":"3efff35d-4ec4-494f-ba03-9264a12e8639","Type":"ContainerStarted","Data":"23fe2bb060ac676323228f42e2fbb10171349689ace7baa0d78e404941166a41"}
Oct 30 00:46:36.690710 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.690686    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5g2l9\""
Oct 30 00:46:36.694286 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.694262    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"]
Oct 30 00:46:36.712315 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.712284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:36.789263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.789213    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hl4nc" podStartSLOduration=1.243560001 podStartE2EDuration="2.789200869s" totalImagesPullingTime="1.545640868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:36.78816739 +0000 UTC m=+304.817477636" watchObservedRunningTime="2026-10-30 00:46:36.789200869 +0000 UTC m=+304.818511098"
Oct 30 00:46:36.808670 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.808629    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnzt2\" (UniqueName: \"kubernetes.io/projected/8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3-kube-api-access-xnzt2\") pod \"authorino-operator-7b7bb4bc77-66ztg\" (UID: \"8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:36.873099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.873067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:36.909353 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.909272    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnzt2\" (UniqueName: \"kubernetes.io/projected/8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3-kube-api-access-xnzt2\") pod \"authorino-operator-7b7bb4bc77-66ztg\" (UID: \"8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:36.950540 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.950503    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnzt2\" (UniqueName: \"kubernetes.io/projected/8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3-kube-api-access-xnzt2\") pod \"authorino-operator-7b7bb4bc77-66ztg\" (UID: \"8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:36.959189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:36.959165    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:37.267868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:37.267819    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"]
Oct 30 00:46:37.269712 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:37.269678    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8377b8b6_6ea1_4d49_9bf2_c57cc68e27d3.slice/crio-3c153c62629cd04774c692a33f8356c051fb4a5d55a307258378be1775efacbf WatchSource:0}: Error finding container 3c153c62629cd04774c692a33f8356c051fb4a5d55a307258378be1775efacbf: Status 404 returned error can't find the container with id 3c153c62629cd04774c692a33f8356c051fb4a5d55a307258378be1775efacbf
Oct 30 00:46:37.678637 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:37.678551    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:37.687975 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:37.687943    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg" event={"ID":"8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3","Type":"ContainerStarted","Data":"3c153c62629cd04774c692a33f8356c051fb4a5d55a307258378be1775efacbf"}
Oct 30 00:46:38.520037 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:38.520006    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:38.669313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:38.669267    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:38.694646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:38.694616    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg" event={"ID":"8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3","Type":"ContainerStarted","Data":"d8da63a0bae3ac7e6447c9876a888a72724c3681e0fff5f473031d6c5d40b12b"}
Oct 30 00:46:38.695066 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:38.694715    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:38.742035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:38.741992    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg" podStartSLOduration=1.461969182 podStartE2EDuration="2.741978743s" totalImagesPullingTime="1.280009561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:38.740459714 +0000 UTC m=+306.769769944" watchObservedRunningTime="2026-10-30 00:46:38.741978743 +0000 UTC m=+306.771288974"
Oct 30 00:46:40.588537 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.588506    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"]
Oct 30 00:46:40.591995 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.591976    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:40.628466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.628446    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jwwj9\""
Oct 30 00:46:40.630395 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.630374    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"]
Oct 30 00:46:40.637999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.637979    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xznxz\" (UniqueName: \"kubernetes.io/projected/338395d0-2720-4943-8adb-cd3a3474526b-kube-api-access-xznxz\") pod \"limitador-operator-controller-manager-5778bb4449-msqjs\" (UID: \"338395d0-2720-4943-8adb-cd3a3474526b\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:40.739007 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.738975    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xznxz\" (UniqueName: \"kubernetes.io/projected/338395d0-2720-4943-8adb-cd3a3474526b-kube-api-access-xznxz\") pod \"limitador-operator-controller-manager-5778bb4449-msqjs\" (UID: \"338395d0-2720-4943-8adb-cd3a3474526b\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:40.782657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.782629    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xznxz\" (UniqueName: \"kubernetes.io/projected/338395d0-2720-4943-8adb-cd3a3474526b-kube-api-access-xznxz\") pod \"limitador-operator-controller-manager-5778bb4449-msqjs\" (UID: \"338395d0-2720-4943-8adb-cd3a3474526b\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:40.848382 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.848295    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:40.904304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:40.904271    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:41.140113 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:41.140036    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:41.207178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:41.207158    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"]
Oct 30 00:46:41.209307 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:41.209279    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338395d0_2720_4943_8adb_cd3a3474526b.slice/crio-74961005ac5c03761ad1f0b536b72a0a890221c237c1ae01490ea3368e2ca1cc WatchSource:0}: Error finding container 74961005ac5c03761ad1f0b536b72a0a890221c237c1ae01490ea3368e2ca1cc: Status 404 returned error can't find the container with id 74961005ac5c03761ad1f0b536b72a0a890221c237c1ae01490ea3368e2ca1cc
Oct 30 00:46:41.710886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:41.710853    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs" event={"ID":"338395d0-2720-4943-8adb-cd3a3474526b","Type":"ContainerStarted","Data":"74961005ac5c03761ad1f0b536b72a0a890221c237c1ae01490ea3368e2ca1cc"}
Oct 30 00:46:43.661542 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:43.661475    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:43.888446 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:43.888383    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:44.015884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.015838    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:44.139529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.139494    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"]
Oct 30 00:46:44.143081 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.143063    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.166448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.166414    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"]
Oct 30 00:46:44.168800 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.168779    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bd6ce02-54e6-45af-9875-a50830402b33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-pwsx2\" (UID: \"1bd6ce02-54e6-45af-9875-a50830402b33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.168936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.168830    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qghbc\" (UniqueName: \"kubernetes.io/projected/1bd6ce02-54e6-45af-9875-a50830402b33-kube-api-access-qghbc\") pod \"kuadrant-operator-controller-manager-678d87766f-pwsx2\" (UID: \"1bd6ce02-54e6-45af-9875-a50830402b33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.269870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.269766    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bd6ce02-54e6-45af-9875-a50830402b33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-pwsx2\" (UID: \"1bd6ce02-54e6-45af-9875-a50830402b33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.269870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.269834    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qghbc\" (UniqueName: \"kubernetes.io/projected/1bd6ce02-54e6-45af-9875-a50830402b33-kube-api-access-qghbc\") pod \"kuadrant-operator-controller-manager-678d87766f-pwsx2\" (UID: \"1bd6ce02-54e6-45af-9875-a50830402b33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.270260 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.270237    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bd6ce02-54e6-45af-9875-a50830402b33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-pwsx2\" (UID: \"1bd6ce02-54e6-45af-9875-a50830402b33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.300265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.300236    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghbc\" (UniqueName: \"kubernetes.io/projected/1bd6ce02-54e6-45af-9875-a50830402b33-kube-api-access-qghbc\") pod \"kuadrant-operator-controller-manager-678d87766f-pwsx2\" (UID: \"1bd6ce02-54e6-45af-9875-a50830402b33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.457563 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.457528    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:44.727302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.727228    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs" event={"ID":"338395d0-2720-4943-8adb-cd3a3474526b","Type":"ContainerStarted","Data":"cd288d5b4b8a0d3a3730dde45be1bf031d5c4eefd9b75ced8d0a0057f6d14c87"}
Oct 30 00:46:44.727644 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.727327    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:44.772377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.772333    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs" podStartSLOduration=1.980441616 podStartE2EDuration="4.772322449s" totalImagesPullingTime="2.791880833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:44.771405094 +0000 UTC m=+312.800715327" watchObservedRunningTime="2026-10-30 00:46:44.772322449 +0000 UTC m=+312.801632679"
Oct 30 00:46:44.787333 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:44.787308    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"]
Oct 30 00:46:44.787746 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:44.787720    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd6ce02_54e6_45af_9875_a50830402b33.slice/crio-522d08b22e3e027173cc22490ef6450cf1887246bff340b1705ff920498d8a26 WatchSource:0}: Error finding container 522d08b22e3e027173cc22490ef6450cf1887246bff340b1705ff920498d8a26: Status 404 returned error can't find the container with id 522d08b22e3e027173cc22490ef6450cf1887246bff340b1705ff920498d8a26
Oct 30 00:46:45.261605 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.261554    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:45.377429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.377390    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:45.677650 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.677564    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nrc8l"
Oct 30 00:46:45.734987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.734958    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2" event={"ID":"1bd6ce02-54e6-45af-9875-a50830402b33","Type":"ContainerStarted","Data":"e5307eaff911b1be57c87c529a78b2d9cc47cf807bb6f5b6846777074bb7ffb5"}
Oct 30 00:46:45.735446 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.735006    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2" event={"ID":"1bd6ce02-54e6-45af-9875-a50830402b33","Type":"ContainerStarted","Data":"522d08b22e3e027173cc22490ef6450cf1887246bff340b1705ff920498d8a26"}
Oct 30 00:46:45.735446 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.735279    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:45.792511 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.792464    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2" podStartSLOduration=1.792449414 podStartE2EDuration="1.792449414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:45.782226501 +0000 UTC m=+313.811536734" watchObservedRunningTime="2026-10-30 00:46:45.792449414 +0000 UTC m=+313.821759651"
Oct 30 00:46:45.858437 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.858401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:45.975086 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:45.975003    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:46.100482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:46.100443    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:46.527037 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:46.527008    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:46.647507 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:46.647472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:46.805795 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:46.805714    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:46.966558 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:46.966513    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:47.406820 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.406788    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:47.627239 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.627211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:47.755396 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.755367    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:47.816353 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.816323    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"]
Oct 30 00:46:47.820189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.820168    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.845380 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.845356    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4vkw8\""
Oct 30 00:46:47.860666 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.860635    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"]
Oct 30 00:46:47.879675 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.879642    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:47.904336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904311    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904490 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904342    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff1714b0-728c-4d2c-8577-c657d2de0c77-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904531 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904494    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904578 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904564    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904613 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904585    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l67hx\" (UniqueName: \"kubernetes.io/projected/ff1714b0-728c-4d2c-8577-c657d2de0c77-kube-api-access-l67hx\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904666 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904649    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904699    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904761 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904748    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:47.904796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:47.904777    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.005657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005577    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.005657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005623    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.005657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005652    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005675    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005693    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff1714b0-728c-4d2c-8577-c657d2de0c77-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005725    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005777    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005804    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l67hx\" (UniqueName: \"kubernetes.io/projected/ff1714b0-728c-4d2c-8577-c657d2de0c77-kube-api-access-l67hx\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.005835    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.006160    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.006229    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006421 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.006338    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006421 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.006410    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.006588 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.006557    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff1714b0-728c-4d2c-8577-c657d2de0c77-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.008160 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.008137    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.008399 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.008384    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.018776 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.018746    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:48.043803 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.043773    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1714b0-728c-4d2c-8577-c657d2de0c77-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.044143 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.044120    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l67hx\" (UniqueName: \"kubernetes.io/projected/ff1714b0-728c-4d2c-8577-c657d2de0c77-kube-api-access-l67hx\") pod \"data-science-gateway-data-science-gateway-class-7c689579c-js4rj\" (UID: \"ff1714b0-728c-4d2c-8577-c657d2de0c77\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.136453 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.136421    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:48.477701 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.477669    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"]
Oct 30 00:46:48.478536 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:46:48.478511    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1714b0_728c_4d2c_8577_c657d2de0c77.slice/crio-083a822fe10eb4ccc82ff355ab7b7409ee67e3d4a155702f41caa39bff9044e8 WatchSource:0}: Error finding container 083a822fe10eb4ccc82ff355ab7b7409ee67e3d4a155702f41caa39bff9044e8: Status 404 returned error can't find the container with id 083a822fe10eb4ccc82ff355ab7b7409ee67e3d4a155702f41caa39bff9044e8
Oct 30 00:46:48.480651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.480620    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:46:48.480741 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.480686    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:46:48.480741 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.480715    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:46:48.496949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.496927    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:48.606258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.606219    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:48.751134 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.751096    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj" event={"ID":"ff1714b0-728c-4d2c-8577-c657d2de0c77","Type":"ContainerStarted","Data":"b13b889609ff3179e8a4976a8ff2a487bd17e061ad34d61cc4e00662c692a9a6"}
Oct 30 00:46:48.751328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.751165    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj" event={"ID":"ff1714b0-728c-4d2c-8577-c657d2de0c77","Type":"ContainerStarted","Data":"083a822fe10eb4ccc82ff355ab7b7409ee67e3d4a155702f41caa39bff9044e8"}
Oct 30 00:46:48.802006 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:48.801956    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj" podStartSLOduration=1.801941751 podStartE2EDuration="1.801941751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:48.799991002 +0000 UTC m=+316.829301231" watchObservedRunningTime="2026-10-30 00:46:48.801941751 +0000 UTC m=+316.831251984"
Oct 30 00:46:49.032018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.031946    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:49.137307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.137269    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:49.142482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.142459    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:49.344063 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.343990    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:49.701783 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.701707    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-66ztg"
Oct 30 00:46:49.758019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.757991    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:49.765003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:49.764977    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-js4rj"
Oct 30 00:46:50.481860 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:50.481819    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:53.065559 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:53.065529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:53.180162 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:53.180126    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:55.737144 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:55.737116    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-msqjs"
Oct 30 00:46:56.743080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:56.743047    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-pwsx2"
Oct 30 00:46:56.837896 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:56.837867    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"]
Oct 30 00:46:56.838093 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:56.838070    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6" podUID="2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" containerName="manager" containerID="cri-o://da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789" gracePeriod=10
Oct 30 00:46:57.012392 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:46:57.012360    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:46:57.084994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.084972    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:57.188889 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.188834    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv\" (UniqueName: \"kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv\") pod \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") "
Oct 30 00:46:57.188889 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.188893    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume\") pod \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\" (UID: \"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa\") "
Oct 30 00:46:57.189404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.189370    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume" pod "2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" (UID: "2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:57.191225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.191202    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv" pod "2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" (UID: "2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa"). InnerVolumeSpecName "kube-api-access-96ntv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:57.290275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.290203    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96ntv\" (UniqueName: \"kubernetes.io/projected/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-kube-api-access-96ntv\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:57.290275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.290227    4577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa-extensions-socket-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:46:57.795924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.795892    4577 generic.go:356] "Generic (PLEG): container finished" podID="2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" containerID="da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789" exitCode=0
Oct 30 00:46:57.796387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.795940    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"
Oct 30 00:46:57.796387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.795985    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6" event={"ID":"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa","Type":"ContainerDied","Data":"da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789"}
Oct 30 00:46:57.796387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.796050    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6" event={"ID":"2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa","Type":"ContainerDied","Data":"29e7b28c116053461bde15b19be6bbdff92cd52f30a913d3382d24e515500c0a"}
Oct 30 00:46:57.796387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.796084    4577 scope.go:122] "RemoveContainer" containerID="da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789"
Oct 30 00:46:57.805331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.805314    4577 scope.go:122] "RemoveContainer" containerID="da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789"
Oct 30 00:46:57.805623 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:46:57.805606    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789\": container with ID starting with da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789 not found: ID does not exist" containerID="da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789"
Oct 30 00:46:57.805697 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.805631    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789"} err="failed to get container status \"da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789\": rpc error: code = NotFound desc = could not find container \"da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789\": container with ID starting with da96b5a706e5f451291c793128ed6821950f56f5d7ee8b7717c322ecd0ae4789 not found: ID does not exist"
Oct 30 00:46:57.855823 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.855800    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"]
Oct 30 00:46:57.869518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:57.869495    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5dcf6"]
Oct 30 00:46:58.109578 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:58.109504    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" path="/var/lib/kubelet/pods/2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa/volumes"
Oct 30 00:46:58.874862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:58.874807    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:46:58.987408 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:46:58.987379    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:01.864717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:01.864687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:01.982001 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:01.981973    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:06.728899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:06.728867    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:06.843459 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:06.843428    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:13.210953 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:13.210922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:13.321077 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:13.321036    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:16.049157 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.049121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:16.160100 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.160069    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:16.695893 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.695858    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:16.710518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.710488    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:16.710904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.710889    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" containerName="manager"
Oct 30 00:47:16.710904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.710905    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" containerName="manager"
Oct 30 00:47:16.711001 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.710982    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f62fb6c-f30e-4b2a-9a50-26aa470ef5aa" containerName="manager"
Oct 30 00:47:16.714883 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.714865    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:16.746219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.746189    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5d5sd\""
Oct 30 00:47:16.746219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.746190    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 00:47:16.749263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.749236    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:16.817952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.817835    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:16.836799 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.836770    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:16.873597 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.873571    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file\") pod \"limitador-limitador-74dfcfb894-mbp7z\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:16.873750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.873606    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptmmx\" (UniqueName: \"kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx\") pod \"limitador-limitador-74dfcfb894-mbp7z\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:16.930272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.930242    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:16.974737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.974662    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file\") pod \"limitador-limitador-74dfcfb894-mbp7z\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:16.974737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.974704    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptmmx\" (UniqueName: \"kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx\") pod \"limitador-limitador-74dfcfb894-mbp7z\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:16.975375 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.975354    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file\") pod \"limitador-limitador-74dfcfb894-mbp7z\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:16.999867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:16.999816    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptmmx\" (UniqueName: \"kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx\") pod \"limitador-limitador-74dfcfb894-mbp7z\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:17.028491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.028468    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:17.330113 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.330088    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:17.331875 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:47:17.331824    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ed64ea_7ffa_4c59_be0d_3760d02652ec.slice/crio-680fda93fff7812dd1ed11a9f7960fa393215ccae55f7c14e7c79bb850eb4af9 WatchSource:0}: Error finding container 680fda93fff7812dd1ed11a9f7960fa393215ccae55f7c14e7c79bb850eb4af9: Status 404 returned error can't find the container with id 680fda93fff7812dd1ed11a9f7960fa393215ccae55f7c14e7c79bb850eb4af9
Oct 30 00:47:17.583616 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.583536    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-m2zlk"]
Oct 30 00:47:17.588902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.588879    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:17.607313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.607287    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-m2zlk"]
Oct 30 00:47:17.614805 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.614784    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2xt6n\""
Oct 30 00:47:17.667203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.667169    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:17.683409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.683373    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzr4b\" (UniqueName: \"kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b\") pod \"authorino-57f7f8659f-m2zlk\" (UID: \"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0\") " pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:17.776206 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.776165    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:17.784288 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.784256    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzr4b\" (UniqueName: \"kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b\") pod \"authorino-57f7f8659f-m2zlk\" (UID: \"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0\") " pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:17.809487 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.809453    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzr4b\" (UniqueName: \"kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b\") pod \"authorino-57f7f8659f-m2zlk\" (UID: \"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0\") " pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:17.895062 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.894968    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z" event={"ID":"84ed64ea-7ffa-4c59-be0d-3760d02652ec","Type":"ContainerStarted","Data":"680fda93fff7812dd1ed11a9f7960fa393215ccae55f7c14e7c79bb850eb4af9"}
Oct 30 00:47:17.909202 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:17.909173    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:18.183556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:18.183480    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:18.218830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:18.218718    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-m2zlk"]
Oct 30 00:47:18.221115 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:47:18.221074    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod013a7d6e_d2bf_4584_82d2_ee8b29cc31f0.slice/crio-ed3cfcf3d598fa23585438c94a45a4337c030d99d32acf9bfea25c8eed94810d WatchSource:0}: Error finding container ed3cfcf3d598fa23585438c94a45a4337c030d99d32acf9bfea25c8eed94810d: Status 404 returned error can't find the container with id ed3cfcf3d598fa23585438c94a45a4337c030d99d32acf9bfea25c8eed94810d
Oct 30 00:47:18.900620 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:18.900584    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-m2zlk" event={"ID":"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0","Type":"ContainerStarted","Data":"ed3cfcf3d598fa23585438c94a45a4337c030d99d32acf9bfea25c8eed94810d"}
Oct 30 00:47:18.902062 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:18.902043    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z" event={"ID":"84ed64ea-7ffa-4c59-be0d-3760d02652ec","Type":"ContainerStarted","Data":"4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7"}
Oct 30 00:47:18.902201 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:18.902189    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:18.947912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:18.947835    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z" podStartSLOduration=1.982079455 podStartE2EDuration="2.947821999s" totalImagesPullingTime="965.742544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:18.945663382 +0000 UTC m=+346.974973613" watchObservedRunningTime="2026-10-30 00:47:18.947821999 +0000 UTC m=+346.977132228"
Oct 30 00:47:19.763872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:19.763820    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:19.926489 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:19.926458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:20.913415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:20.913374    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-m2zlk" event={"ID":"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0","Type":"ContainerStarted","Data":"582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e"}
Oct 30 00:47:20.955860 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:20.955788    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-m2zlk" podStartSLOduration=2.053325481 podStartE2EDuration="3.955773452s" totalImagesPullingTime="1.902447971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:20.95458391 +0000 UTC m=+348.983894336" watchObservedRunningTime="2026-10-30 00:47:20.955773452 +0000 UTC m=+348.985083681"
Oct 30 00:47:21.954612 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:21.954584    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:22.035688 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:22.035657    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-m2zlk"]
Oct 30 00:47:22.068980 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:22.068949    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:22.755879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:22.755827    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:22.923559 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:22.923524    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-m2zlk" podUID="013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" containerName="authorino" containerID="cri-o://582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e" gracePeriod=30
Oct 30 00:47:22.928241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:22.928215    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:23.171714 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.171691    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:23.231194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.231164    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b\" (UniqueName: \"kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b\") pod \"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0\" (UID: \"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0\") "
Oct 30 00:47:23.233485 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.233452    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b" pod "013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" (UID: "013a7d6e-d2bf-4584-82d2-ee8b29cc31f0"). InnerVolumeSpecName "kube-api-access-kzr4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:47:23.332132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.332041    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzr4b\" (UniqueName: \"kubernetes.io/projected/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0-kube-api-access-kzr4b\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:47:23.929980 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.929945    4577 generic.go:356] "Generic (PLEG): container finished" podID="013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" containerID="582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e" exitCode=0
Oct 30 00:47:23.930156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.929991    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-m2zlk"
Oct 30 00:47:23.930156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.930042    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-m2zlk" event={"ID":"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0","Type":"ContainerDied","Data":"582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e"}
Oct 30 00:47:23.930156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.930091    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-m2zlk" event={"ID":"013a7d6e-d2bf-4584-82d2-ee8b29cc31f0","Type":"ContainerDied","Data":"ed3cfcf3d598fa23585438c94a45a4337c030d99d32acf9bfea25c8eed94810d"}
Oct 30 00:47:23.930156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.930113    4577 scope.go:122] "RemoveContainer" containerID="582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e"
Oct 30 00:47:23.938883 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.938865    4577 scope.go:122] "RemoveContainer" containerID="582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e"
Oct 30 00:47:23.939147 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:47:23.939126    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e\": container with ID starting with 582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e not found: ID does not exist" containerID="582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e"
Oct 30 00:47:23.939211 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.939156    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e"} err="failed to get container status \"582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e\": rpc error: code = NotFound desc = could not find container \"582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e\": container with ID starting with 582ae33e496b3b7606d89fc4c7b58849cb27e46ec3531d1a60e81d879cc6998e not found: ID does not exist"
Oct 30 00:47:23.985412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.985387    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-m2zlk"]
Oct 30 00:47:23.989502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:23.989478    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-m2zlk"]
Oct 30 00:47:24.109080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:24.109051    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" path="/var/lib/kubelet/pods/013a7d6e-d2bf-4584-82d2-ee8b29cc31f0/volumes"
Oct 30 00:47:27.011814 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:47:27.011776    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:47:27.962610 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:27.962571    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:28.077053 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:28.077028    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:28.363248 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:28.363216    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:28.474565 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:28.474523    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:28.586558 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:28.586508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:29.908707 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:29.908679    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:33.008548 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.008520    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:33.008976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.008697    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z" podUID="84ed64ea-7ffa-4c59-be0d-3760d02652ec" containerName="limitador" containerID="cri-o://4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7" gracePeriod=30
Oct 30 00:47:33.546791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.546765    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:33.619898 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.619809    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx\" (UniqueName: \"kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx\") pod \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") "
Oct 30 00:47:33.619898 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.619873    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file\" (UniqueName: \"kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file\") pod \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\" (UID: \"84ed64ea-7ffa-4c59-be0d-3760d02652ec\") "
Oct 30 00:47:33.620256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.620236    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file" pod "84ed64ea-7ffa-4c59-be0d-3760d02652ec" (UID: "84ed64ea-7ffa-4c59-be0d-3760d02652ec"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:47:33.621952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.621925    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx" pod "84ed64ea-7ffa-4c59-be0d-3760d02652ec" (UID: "84ed64ea-7ffa-4c59-be0d-3760d02652ec"). InnerVolumeSpecName "kube-api-access-ptmmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:47:33.720716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.720686    4577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84ed64ea-7ffa-4c59-be0d-3760d02652ec-config-file\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:47:33.720716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.720713    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptmmx\" (UniqueName: \"kubernetes.io/projected/84ed64ea-7ffa-4c59-be0d-3760d02652ec-kube-api-access-ptmmx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:47:33.983422 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.983384    4577 generic.go:356] "Generic (PLEG): container finished" podID="84ed64ea-7ffa-4c59-be0d-3760d02652ec" containerID="4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7" exitCode=0
Oct 30 00:47:33.983588 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.983431    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"
Oct 30 00:47:33.983588 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.983430    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z" event={"ID":"84ed64ea-7ffa-4c59-be0d-3760d02652ec","Type":"ContainerDied","Data":"4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7"}
Oct 30 00:47:33.983588 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.983570    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-mbp7z" event={"ID":"84ed64ea-7ffa-4c59-be0d-3760d02652ec","Type":"ContainerDied","Data":"680fda93fff7812dd1ed11a9f7960fa393215ccae55f7c14e7c79bb850eb4af9"}
Oct 30 00:47:33.983684 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.983599    4577 scope.go:122] "RemoveContainer" containerID="4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7"
Oct 30 00:47:33.992680 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.992660    4577 scope.go:122] "RemoveContainer" containerID="4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7"
Oct 30 00:47:33.992976 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:47:33.992948    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7\": container with ID starting with 4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7 not found: ID does not exist" containerID="4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7"
Oct 30 00:47:33.993043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:33.992984    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7"} err="failed to get container status \"4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7\": rpc error: code = NotFound desc = could not find container \"4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7\": container with ID starting with 4bbb2c25ee3676fc50cd1488bb8dbe0076ba9160d8ef8e796e0ba4229dcc43f7 not found: ID does not exist"
Oct 30 00:47:34.040590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:34.040567    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:34.054171 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:34.054138    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-mbp7z"]
Oct 30 00:47:34.109735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:34.109705    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84ed64ea-7ffa-4c59-be0d-3760d02652ec" path="/var/lib/kubelet/pods/84ed64ea-7ffa-4c59-be0d-3760d02652ec/volumes"
Oct 30 00:47:35.339412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:35.339382    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:35.481038 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:35.480996    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:35.637594 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:35.637518    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:35.748942 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:35.748913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:36.052008 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.051977    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"]
Oct 30 00:47:36.052325 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.052312    4577 cpu_manager.go:414] "containerMap: removing container" podUID="013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" containerName="authorino"
Oct 30 00:47:36.052325 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.052324    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" containerName="authorino"
Oct 30 00:47:36.052470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.052345    4577 cpu_manager.go:414] "containerMap: removing container" podUID="84ed64ea-7ffa-4c59-be0d-3760d02652ec" containerName="limitador"
Oct 30 00:47:36.052470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.052350    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84ed64ea-7ffa-4c59-be0d-3760d02652ec" containerName="limitador"
Oct 30 00:47:36.052470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.052408    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="84ed64ea-7ffa-4c59-be0d-3760d02652ec" containerName="limitador"
Oct 30 00:47:36.052470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.052419    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="013a7d6e-d2bf-4584-82d2-ee8b29cc31f0" containerName="authorino"
Oct 30 00:47:36.056816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.056794    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.073501 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.073480    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 00:47:36.087264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.087242    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 00:47:36.087397 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.087266    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-x9fmt\""
Oct 30 00:47:36.089231 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.089212    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"]
Oct 30 00:47:36.124955 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.124930    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:36.150334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.150309    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5cef7ef0-fef6-47df-8a73-531458d5a43e-data\") pod \"postgres-7cdbf6dc44-nqbth\" (UID: \"5cef7ef0-fef6-47df-8a73-531458d5a43e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.150586 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.150568    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wbpf\" (UniqueName: \"kubernetes.io/projected/5cef7ef0-fef6-47df-8a73-531458d5a43e-kube-api-access-4wbpf\") pod \"postgres-7cdbf6dc44-nqbth\" (UID: \"5cef7ef0-fef6-47df-8a73-531458d5a43e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.173748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.173725    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 00:47:36.251520 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.251482    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5cef7ef0-fef6-47df-8a73-531458d5a43e-data\") pod \"postgres-7cdbf6dc44-nqbth\" (UID: \"5cef7ef0-fef6-47df-8a73-531458d5a43e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.251703 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.251571    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wbpf\" (UniqueName: \"kubernetes.io/projected/5cef7ef0-fef6-47df-8a73-531458d5a43e-kube-api-access-4wbpf\") pod \"postgres-7cdbf6dc44-nqbth\" (UID: \"5cef7ef0-fef6-47df-8a73-531458d5a43e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.251859 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.251823    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5cef7ef0-fef6-47df-8a73-531458d5a43e-data\") pod \"postgres-7cdbf6dc44-nqbth\" (UID: \"5cef7ef0-fef6-47df-8a73-531458d5a43e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.276633 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.276609    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wbpf\" (UniqueName: \"kubernetes.io/projected/5cef7ef0-fef6-47df-8a73-531458d5a43e-kube-api-access-4wbpf\") pod \"postgres-7cdbf6dc44-nqbth\" (UID: \"5cef7ef0-fef6-47df-8a73-531458d5a43e\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.370423 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.370346    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:36.674732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.674708    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"]
Oct 30 00:47:36.676445 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:47:36.676412    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cef7ef0_fef6_47df_8a73_531458d5a43e.slice/crio-fb1ae2243f98ac40daf3e9fe41323484c52c5d5e31bfb1cfb7d62dd4dddf71bf WatchSource:0}: Error finding container fb1ae2243f98ac40daf3e9fe41323484c52c5d5e31bfb1cfb7d62dd4dddf71bf: Status 404 returned error can't find the container with id fb1ae2243f98ac40daf3e9fe41323484c52c5d5e31bfb1cfb7d62dd4dddf71bf
Oct 30 00:47:36.999674 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:36.999641    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth" event={"ID":"5cef7ef0-fef6-47df-8a73-531458d5a43e","Type":"ContainerStarted","Data":"fb1ae2243f98ac40daf3e9fe41323484c52c5d5e31bfb1cfb7d62dd4dddf71bf"}
Oct 30 00:47:39.177690 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:39.177653    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:41.388419 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:41.388114    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:41.584200 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:41.584159    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:42.029137 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:42.029103    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth" event={"ID":"5cef7ef0-fef6-47df-8a73-531458d5a43e","Type":"ContainerStarted","Data":"63f9421612ae31b51366b0b175dc37dda78bd95dfbac1eb79991f9dd01d477bf"}
Oct 30 00:47:42.029304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:42.029249    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:42.074101 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:42.074048    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth" podStartSLOduration=1.396874191 podStartE2EDuration="6.074034085s" totalImagesPullingTime="4.677159894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:42.072258983 +0000 UTC m=+370.101569213" watchObservedRunningTime="2026-10-30 00:47:42.074034085 +0000 UTC m=+370.103344315"
Oct 30 00:47:44.423448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:44.423409    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:44.571816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:44.571785    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:48.064150 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:48.064125    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nqbth"
Oct 30 00:47:50.804066 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.804031    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"]
Oct 30 00:47:50.819824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.819800    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.833120 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.833091    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"]
Oct 30 00:47:50.844429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.844396    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 00:47:50.844429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.844429    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 00:47:50.844748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.844662    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-htf9g\""
Oct 30 00:47:50.844748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.844723    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 00:47:50.884130 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884105    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d5e-0573-4331-93a9-15205495cb83-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.884245 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884136    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.884245 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884155    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93475d5e-0573-4331-93a9-15205495cb83-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.884245 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884174    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvgdc\" (UniqueName: \"kubernetes.io/projected/93475d5e-0573-4331-93a9-15205495cb83-kube-api-access-rvgdc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.884245 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884225    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.884377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884340    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.884409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.884391    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d5e-0573-4331-93a9-15205495cb83-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.984820 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.984783    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.984869    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d5e-0573-4331-93a9-15205495cb83-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.984919    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d5e-0573-4331-93a9-15205495cb83-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.984935    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.984952    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93475d5e-0573-4331-93a9-15205495cb83-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.984969    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvgdc\" (UniqueName: \"kubernetes.io/projected/93475d5e-0573-4331-93a9-15205495cb83-kube-api-access-rvgdc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.985046    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.985785 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.985755    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/93475d5e-0573-4331-93a9-15205495cb83-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.987367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.987338    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d5e-0573-4331-93a9-15205495cb83-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.987529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.987513    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.987768 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.987752    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:50.987874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:50.987826    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/93475d5e-0573-4331-93a9-15205495cb83-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:51.012154 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.012133    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d5e-0573-4331-93a9-15205495cb83-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:51.012373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.012354    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvgdc\" (UniqueName: \"kubernetes.io/projected/93475d5e-0573-4331-93a9-15205495cb83-kube-api-access-rvgdc\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f8t97\" (UID: \"93475d5e-0573-4331-93a9-15205495cb83\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:51.143982 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.143913    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:51.488276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.488248    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"]
Oct 30 00:47:51.491489 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:47:51.491456    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93475d5e_0573_4331_93a9_15205495cb83.slice/crio-c4b09e5021114e41a473f43a5ea0743ec48937859a9fd8b504d62974ed528641 WatchSource:0}: Error finding container c4b09e5021114e41a473f43a5ea0743ec48937859a9fd8b504d62974ed528641: Status 404 returned error can't find the container with id c4b09e5021114e41a473f43a5ea0743ec48937859a9fd8b504d62974ed528641
Oct 30 00:47:51.599746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.599707    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:51.711907 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.711874    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:51.825031 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.824957    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:51.953475 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:51.953421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:52.074661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:52.074630    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:52.083075 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:52.083003    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97" event={"ID":"93475d5e-0573-4331-93a9-15205495cb83","Type":"ContainerStarted","Data":"c4b09e5021114e41a473f43a5ea0743ec48937859a9fd8b504d62974ed528641"}
Oct 30 00:47:52.389556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:52.389482    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:54.368387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:54.368358    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:54.486018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:54.485989    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:54.596137 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:54.596111    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:54.710717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:54.710649    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:54.829720 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:54.829686    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:55.493249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.493211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:55.573864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.573821    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-cd686c5b8-qggw6"]
Oct 30 00:47:55.580467 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.580446    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.600203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.600176    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-cd686c5b8-qggw6"]
Oct 30 00:47:55.611914 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.611873    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 00:47:55.612063 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.611870    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:55.612063 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.612052    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 00:47:55.612178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.612094    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 00:47:55.614772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.614759    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 00:47:55.614873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.614835    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-gzqd4\""
Oct 30 00:47:55.614973 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.614960    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 00:47:55.722036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.722001    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:55.725013 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.724992    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.725122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.725035    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxtrd\" (UniqueName: \"kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.725122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.725054    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.825952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.825874    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.825952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.825923    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxtrd\" (UniqueName: \"kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.825952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.825941    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.828382 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.828358    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.828488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.828434    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.831003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.830983    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:55.850583 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.850556    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxtrd\" (UniqueName: \"kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd\") pod \"maas-controller-cd686c5b8-qggw6\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") " pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:55.893598 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:55.893577    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:47:56.188394 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:56.188373    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-cd686c5b8-qggw6"]
Oct 30 00:47:56.190935 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:47:56.190907    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41270f9e_1e0e_40e4_9c26_f1e0f2ff8c78.slice/crio-5f4cdab871b28ec8c0c2a9bf9887c9a55148f2852a008b3026985218d41536b7 WatchSource:0}: Error finding container 5f4cdab871b28ec8c0c2a9bf9887c9a55148f2852a008b3026985218d41536b7: Status 404 returned error can't find the container with id 5f4cdab871b28ec8c0c2a9bf9887c9a55148f2852a008b3026985218d41536b7
Oct 30 00:47:57.026038 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:47:57.025995    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:47:57.111725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:57.111686    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cd686c5b8-qggw6" event={"ID":"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78","Type":"ContainerStarted","Data":"5f4cdab871b28ec8c0c2a9bf9887c9a55148f2852a008b3026985218d41536b7"}
Oct 30 00:47:58.221982 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:58.221946    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:47:58.222312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:58.222019    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:47:59.110383 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.110312    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:59.127113 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.127080    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97" event={"ID":"93475d5e-0573-4331-93a9-15205495cb83","Type":"ContainerStarted","Data":"d15fef98616c7823a673da892519d754e27cd124c815d928097bc12b44de7e98"}
Oct 30 00:47:59.127279 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.127235    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:47:59.128838 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.128808    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cd686c5b8-qggw6" event={"ID":"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78","Type":"ContainerStarted","Data":"08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c"}
Oct 30 00:47:59.129651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.129622    4577 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-f8t97 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 00:47:59.129766 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.129673    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97" podUID="93475d5e-0573-4331-93a9-15205495cb83" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 00:47:59.225101 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.225069    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:47:59.239066 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.239021    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-cd686c5b8-qggw6" podStartSLOduration=1.628907208 podStartE2EDuration="4.239007487s" totalImagesPullingTime="2.610100279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:59.237463346 +0000 UTC m=+387.266773576" watchObservedRunningTime="2026-10-30 00:47:59.239007487 +0000 UTC m=+387.268317707"
Oct 30 00:47:59.239437 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:47:59.239415    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97" podStartSLOduration=2.51192436 podStartE2EDuration="9.239408423s" totalImagesPullingTime="6.727484063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:59.189320945 +0000 UTC m=+387.218631174" watchObservedRunningTime="2026-10-30 00:47:59.239408423 +0000 UTC m=+387.268718660"
Oct 30 00:48:00.132960 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:00.132935    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:48:00.133636 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:00.133620    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f8t97"
Oct 30 00:48:01.811032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:01.810998    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:01.925357 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:01.925326    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:11.140222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.140188    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:48:11.219495 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.219459    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-cd686c5b8-qggw6"]
Oct 30 00:48:11.219710 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.219687    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-cd686c5b8-qggw6" podUID="41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" containerName="manager" containerID="cri-o://08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c" gracePeriod=30
Oct 30 00:48:11.461103 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.461077    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:48:11.469310 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.469288    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert\") pod \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") "
Oct 30 00:48:11.469418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.469324    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs\") pod \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") "
Oct 30 00:48:11.469418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.469409    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd\" (UniqueName: \"kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd\") pod \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\" (UID: \"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78\") "
Oct 30 00:48:11.471649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.471628    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert" pod "41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" (UID: "41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:48:11.471750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.471643    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs" pod "41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" (UID: "41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:48:11.471750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.471642    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd" pod "41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" (UID: "41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78"). InnerVolumeSpecName "kube-api-access-rxtrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:48:11.570071 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.570032    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxtrd\" (UniqueName: \"kubernetes.io/projected/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-kube-api-access-rxtrd\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:48:11.570071 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.570066    4577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-webhook-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:48:11.570071 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:11.570076    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:48:12.195029 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.194996    4577 generic.go:356] "Generic (PLEG): container finished" podID="41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" containerID="08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c" exitCode=0
Oct 30 00:48:12.195494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.195041    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cd686c5b8-qggw6"
Oct 30 00:48:12.195494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.195083    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cd686c5b8-qggw6" event={"ID":"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78","Type":"ContainerDied","Data":"08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c"}
Oct 30 00:48:12.195494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.195142    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cd686c5b8-qggw6" event={"ID":"41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78","Type":"ContainerDied","Data":"5f4cdab871b28ec8c0c2a9bf9887c9a55148f2852a008b3026985218d41536b7"}
Oct 30 00:48:12.195494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.195176    4577 scope.go:122] "RemoveContainer" containerID="08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c"
Oct 30 00:48:12.203776 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.203761    4577 scope.go:122] "RemoveContainer" containerID="08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c"
Oct 30 00:48:12.204054 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:48:12.204036    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c\": container with ID starting with 08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c not found: ID does not exist" containerID="08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c"
Oct 30 00:48:12.204116 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.204061    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c"} err="failed to get container status \"08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c\": rpc error: code = NotFound desc = could not find container \"08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c\": container with ID starting with 08215da20de7d6eb33ed286ab9409d5a533e9bbc3c7d8abb6d73421edc03ec4c not found: ID does not exist"
Oct 30 00:48:12.254932 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.254906    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-cd686c5b8-qggw6"]
Oct 30 00:48:12.268999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:12.268973    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-cd686c5b8-qggw6"]
Oct 30 00:48:14.109399 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:14.109313    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" path="/var/lib/kubelet/pods/41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78/volumes"
Oct 30 00:48:18.315027 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:18.314992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:27.014134 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:48:27.014091    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:48:30.308159 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:30.308120    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:30.436924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:30.436891    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:31.753756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:31.753715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:31.867562 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:31.867528    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:35.225019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:35.224983    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:42.048951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.048918    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:42.158433 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.158405    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:48:42.593463 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.593414    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-hp7kp"]
Oct 30 00:48:42.593795 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.593781    4577 cpu_manager.go:414] "containerMap: removing container" podUID="41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" containerName="manager"
Oct 30 00:48:42.593795 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.593795    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" containerName="manager"
Oct 30 00:48:42.593906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.593887    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="41270f9e-1e0e-40e4-9c26-f1e0f2ff8c78" containerName="manager"
Oct 30 00:48:42.596215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.596191    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:42.614819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.614796    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-hp7kp"]
Oct 30 00:48:42.626362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.626344    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2xt6n\""
Oct 30 00:48:42.626481 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.626468    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 00:48:42.643918 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.643897    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpt29\" (UniqueName: \"kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29\") pod \"authorino-6889cbd8c4-hp7kp\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") " pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:42.644011 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.643959    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert\") pod \"authorino-6889cbd8c4-hp7kp\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") " pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:42.712945 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.712912    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-hp7kp"]
Oct 30 00:48:42.714032 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:48:42.713138    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qpt29 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-hp7kp" podUID="36079fa5-601f-402c-8fce-4da12b30059e"
Oct 30 00:48:42.745275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.745243    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpt29\" (UniqueName: \"kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29\") pod \"authorino-6889cbd8c4-hp7kp\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") " pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:42.745444 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.745309    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert\") pod \"authorino-6889cbd8c4-hp7kp\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") " pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:42.747873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.747837    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert\") pod \"authorino-6889cbd8c4-hp7kp\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") " pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:42.781335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:42.781308    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpt29\" (UniqueName: \"kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29\") pod \"authorino-6889cbd8c4-hp7kp\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") " pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:43.347746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.347714    4577 generic.go:356] "Generic (PLEG): container finished" podID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerID="b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244" exitCode=0
Oct 30 00:48:43.348158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.347789    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" event={"ID":"46950bd3-baa1-4289-bc9c-0a85b6548788","Type":"ContainerDied","Data":"b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"}
Oct 30 00:48:43.348158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.347898    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:43.348325 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.348311    4577 scope.go:122] "RemoveContainer" containerID="b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"
Oct 30 00:48:43.349635 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.349602    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:48:43.349729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.349690    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:48:43.349769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.349738    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:48:43.354402 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.354381    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:43.451651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.451618    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29\" (UniqueName: \"kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29\") pod \"36079fa5-601f-402c-8fce-4da12b30059e\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") "
Oct 30 00:48:43.451830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.451660    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert\" (UniqueName: \"kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert\") pod \"36079fa5-601f-402c-8fce-4da12b30059e\" (UID: \"36079fa5-601f-402c-8fce-4da12b30059e\") "
Oct 30 00:48:43.455486 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.455458    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29" pod "36079fa5-601f-402c-8fce-4da12b30059e" (UID: "36079fa5-601f-402c-8fce-4da12b30059e"). InnerVolumeSpecName "kube-api-access-qpt29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:48:43.455791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.455769    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert" pod "36079fa5-601f-402c-8fce-4da12b30059e" (UID: "36079fa5-601f-402c-8fce-4da12b30059e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:48:43.553636 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.553590    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpt29\" (UniqueName: \"kubernetes.io/projected/36079fa5-601f-402c-8fce-4da12b30059e-kube-api-access-qpt29\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:48:43.553636 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:43.553639    4577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/36079fa5-601f-402c-8fce-4da12b30059e-tls-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:48:44.375963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.375877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" event={"ID":"46950bd3-baa1-4289-bc9c-0a85b6548788","Type":"ContainerStarted","Data":"81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a"}
Oct 30 00:48:44.375963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.375947    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-hp7kp"
Oct 30 00:48:44.510980 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.510936    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-hp7kp"]
Oct 30 00:48:44.514868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.514826    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-hp7kp"]
Oct 30 00:48:44.979221 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.979182    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:48:44.979455 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.979433    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-ljrb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Oct 30 00:48:44.979535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:44.979489    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Oct 30 00:48:45.979321 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:45.979284    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-ljrb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Oct 30 00:48:45.979717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:45.979341    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Oct 30 00:48:46.109857 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:46.109820    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36079fa5-601f-402c-8fce-4da12b30059e" path="/var/lib/kubelet/pods/36079fa5-601f-402c-8fce-4da12b30059e/volumes"
Oct 30 00:48:46.978940 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:46.978901    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-ljrb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Oct 30 00:48:46.979123 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:46.978957    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Oct 30 00:48:47.978994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:47.978951    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-ljrb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Oct 30 00:48:47.979389 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:47.979013    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Oct 30 00:48:48.979105 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:48.979065    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-ljrb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Oct 30 00:48:48.979502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:48.979132    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Oct 30 00:48:49.982780 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:49.982752    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:48:49.983277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:49.983160    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:48:49.983783 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:48:49.983765    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:48:57.011649 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:48:57.011613    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:49:18.445246 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:18.445207    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:18.558016 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:18.557984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:19.707037 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:19.707001    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:19.907378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:19.907349    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:23.967029 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:23.966983    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:24.077653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:24.077616    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:27.012520 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:49:27.012479    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:49:39.099040 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:39.099006    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:48.621774 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:48.621732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:49.127201 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:49.127158    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:49.717575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:49.717548    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:49.826991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:49.826959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:50.454121 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:50.454084    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:50.569389 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:50.569333    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:54.766733 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:54.766693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:49:57.013575 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:49:57.013534    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:49:58.673460 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:49:58.673426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:01.724909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:01.724868    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:03.223440 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.223404    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"]
Oct 30 00:50:03.226059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.226033    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.250426 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.250397    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pmws7\""
Oct 30 00:50:03.253362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.253333    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"]
Oct 30 00:50:03.264340 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.264311    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 00:50:03.264340 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.264317    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 00:50:03.264507 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.264484    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 00:50:03.285912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.285884    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:03.290989 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.290968    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.291093 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.291008    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.291093 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.291076    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fr7b\" (UniqueName: \"kubernetes.io/projected/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-kube-api-access-6fr7b\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.291178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.291111    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.291178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.291153    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.291307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.291199    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.391985    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392056    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392078    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392213    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392275    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392319    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fr7b\" (UniqueName: \"kubernetes.io/projected/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-kube-api-access-6fr7b\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392583    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392759 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392737    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.392963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.392771    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.394633 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.394609    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.394877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.394858    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.419705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.419668    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fr7b\" (UniqueName: \"kubernetes.io/projected/f0f0cce1-38e9-4093-a90f-bee3ad2b674f-kube-api-access-6fr7b\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-nw82l\" (UID: \"f0f0cce1-38e9-4093-a90f-bee3ad2b674f\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.538044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.538011    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:03.837868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:03.837812    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"]
Oct 30 00:50:03.839919 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:50:03.839890    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f0cce1_38e9_4093_a90f_bee3ad2b674f.slice/crio-7dfb74706750965b02e2af034582948b941070a7ae41c82bdc577ce07c83dcce WatchSource:0}: Error finding container 7dfb74706750965b02e2af034582948b941070a7ae41c82bdc577ce07c83dcce: Status 404 returned error can't find the container with id 7dfb74706750965b02e2af034582948b941070a7ae41c82bdc577ce07c83dcce
Oct 30 00:50:04.766135 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.766097    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l" event={"ID":"f0f0cce1-38e9-4093-a90f-bee3ad2b674f","Type":"ContainerStarted","Data":"7dfb74706750965b02e2af034582948b941070a7ae41c82bdc577ce07c83dcce"}
Oct 30 00:50:04.767056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.767033    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"]
Oct 30 00:50:04.769920 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.769894    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.790763 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.790732    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"]
Oct 30 00:50:04.801404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.801368    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 00:50:04.807819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.807782    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.807965 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.807929    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.808015 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.807997    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.808121 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.808088    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fmz\" (UniqueName: \"kubernetes.io/projected/e8c30c7a-2aeb-4913-b607-f752f1bb1911-kube-api-access-p2fmz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.808236 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.808138    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8c30c7a-2aeb-4913-b607-f752f1bb1911-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.808236 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.808197    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.839083 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.839044    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:04.909208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909172    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fmz\" (UniqueName: \"kubernetes.io/projected/e8c30c7a-2aeb-4913-b607-f752f1bb1911-kube-api-access-p2fmz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909227    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8c30c7a-2aeb-4913-b607-f752f1bb1911-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909287    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909474    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909588 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909545    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909606    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909775 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909728    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.909943 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.909781    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.910088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.910071    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.912041 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.911996    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8c30c7a-2aeb-4913-b607-f752f1bb1911-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.912356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.912339    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8c30c7a-2aeb-4913-b607-f752f1bb1911-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:04.934295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:04.934269    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fmz\" (UniqueName: \"kubernetes.io/projected/e8c30c7a-2aeb-4913-b607-f752f1bb1911-kube-api-access-p2fmz\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b\" (UID: \"e8c30c7a-2aeb-4913-b607-f752f1bb1911\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:05.089582 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.089343    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:05.396294 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.396268    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"]
Oct 30 00:50:05.398535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:50:05.398397    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c30c7a_2aeb_4913_b607_f752f1bb1911.slice/crio-af8c6c803ecf67717e9bf76f19ce1ed143211337f477543d11447fc16f7993f6 WatchSource:0}: Error finding container af8c6c803ecf67717e9bf76f19ce1ed143211337f477543d11447fc16f7993f6: Status 404 returned error can't find the container with id af8c6c803ecf67717e9bf76f19ce1ed143211337f477543d11447fc16f7993f6
Oct 30 00:50:05.773319 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.773284    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b" event={"ID":"e8c30c7a-2aeb-4913-b607-f752f1bb1911","Type":"ContainerStarted","Data":"a350489092fafaa82f386d838ddf36c1976b4205b715f204782dd6a4389d272f"}
Oct 30 00:50:05.773809 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.773348    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b" event={"ID":"e8c30c7a-2aeb-4913-b607-f752f1bb1911","Type":"ContainerStarted","Data":"af8c6c803ecf67717e9bf76f19ce1ed143211337f477543d11447fc16f7993f6"}
Oct 30 00:50:05.775081 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.775060    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l" event={"ID":"f0f0cce1-38e9-4093-a90f-bee3ad2b674f","Type":"ContainerStarted","Data":"90deb8a15ff88cd269d3d638834188b881b0acd05523482175ad448b9e5c249c"}
Oct 30 00:50:05.824965 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.824920    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b" podStartSLOduration=1.653935347 podStartE2EDuration="1.824906523s" totalImagesPullingTime="170.971176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:05.823795414 +0000 UTC m=+513.853105644" watchObservedRunningTime="2026-10-30 00:50:05.824906523 +0000 UTC m=+513.854216753"
Oct 30 00:50:05.870691 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:05.870643    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l" podStartSLOduration=1.633443814 podStartE2EDuration="2.870629609s" totalImagesPullingTime="1.237185795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:05.869114663 +0000 UTC m=+513.898424893" watchObservedRunningTime="2026-10-30 00:50:05.870629609 +0000 UTC m=+513.899939832"
Oct 30 00:50:06.089562 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.089477    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:06.487470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.487441    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-wc2r5"]
Oct 30 00:50:06.490777 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.490752    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.513164 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.513137    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-wc2r5"]
Oct 30 00:50:06.524093 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.524066    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 00:50:06.526894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.526864    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.527101 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.527077    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.527203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.527149    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2jr5\" (UniqueName: \"kubernetes.io/projected/b48c6744-79c5-46c4-938d-f7f943ea9a16-kube-api-access-n2jr5\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.527273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.527256    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-home\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.527308 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.527286    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.527352 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.527313    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b48c6744-79c5-46c4-938d-f7f943ea9a16-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.538153 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.538130    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:06.567541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.567519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:06.628460 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.628417    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jr5\" (UniqueName: \"kubernetes.io/projected/b48c6744-79c5-46c4-938d-f7f943ea9a16-kube-api-access-n2jr5\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.628653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.628506    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-home\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.628653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.628535    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.628653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.628562    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b48c6744-79c5-46c4-938d-f7f943ea9a16-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.628653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.628645    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.628908 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.628674    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.629191 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.629162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.629529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.629466    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.629633 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.629530    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-home\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.632101 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.632057    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b48c6744-79c5-46c4-938d-f7f943ea9a16-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.635387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.635366    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b48c6744-79c5-46c4-938d-f7f943ea9a16-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.653316 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.653288    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2jr5\" (UniqueName: \"kubernetes.io/projected/b48c6744-79c5-46c4-938d-f7f943ea9a16-kube-api-access-n2jr5\") pod \"echo-simulator-kserve-64b6f8f96c-wc2r5\" (UID: \"b48c6744-79c5-46c4-938d-f7f943ea9a16\") " pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:06.806037 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:06.805913    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:07.117165 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:07.117138    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-wc2r5"]
Oct 30 00:50:07.119253 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:50:07.119229    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb48c6744_79c5_46c4_938d_f7f943ea9a16.slice/crio-a7ad0f761a7b0d32eeb6bd97251208b5f1130b2473f4edbb535b14d9546ca779 WatchSource:0}: Error finding container a7ad0f761a7b0d32eeb6bd97251208b5f1130b2473f4edbb535b14d9546ca779: Status 404 returned error can't find the container with id a7ad0f761a7b0d32eeb6bd97251208b5f1130b2473f4edbb535b14d9546ca779
Oct 30 00:50:07.790438 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:07.790399    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5" event={"ID":"b48c6744-79c5-46c4-938d-f7f943ea9a16","Type":"ContainerStarted","Data":"a7ad0f761a7b0d32eeb6bd97251208b5f1130b2473f4edbb535b14d9546ca779"}
Oct 30 00:50:08.596632 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:08.596574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:08.797222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:08.797185    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5" event={"ID":"b48c6744-79c5-46c4-938d-f7f943ea9a16","Type":"ContainerStarted","Data":"944d98e8300664fd0d25dea2896301149e181b20931fc7548f745eff9faf6302"}
Oct 30 00:50:08.806637 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:08.806617    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:08.844172 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:08.844129    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5" podStartSLOduration=1.3500479 podStartE2EDuration="2.844116098s" totalImagesPullingTime="1.494068198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:08.842678688 +0000 UTC m=+516.871988917" watchObservedRunningTime="2026-10-30 00:50:08.844116098 +0000 UTC m=+516.873426327"
Oct 30 00:50:11.161862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:11.161795    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:13.539067 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:13.539032    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:13.552104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:13.552074    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:13.837344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:13.837255    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-nw82l"
Oct 30 00:50:15.089819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:15.089782    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:15.104793 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:15.104767    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:15.848066 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:15.848039    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b"
Oct 30 00:50:16.806720 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:16.806677    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:16.820014 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:16.819989    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:16.854697 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:16.854670    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-wc2r5"
Oct 30 00:50:18.654931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:18.654884    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:20.630477 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:20.630443    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:50:27.012147 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:50:27.012113    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:50:27.481195 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.481162    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"]
Oct 30 00:50:27.487978 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.487959    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.524333 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.524305    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"]
Oct 30 00:50:27.644428 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644385    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644428 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644427    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644501    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt5bl\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644551    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644584    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644602    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644630    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644714    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.644835 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.644812    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746285    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746345    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt5bl\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746381    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746420    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746447    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746481    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746506    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746588    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746626    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.746903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.746881    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.747055 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.747031    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.747177 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.747143    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.747305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.747191    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.747305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.747243    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.749229 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.749200    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.749409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.749392    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.771400 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.771372    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.771678 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.771655    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt5bl\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl\") pod \"maas-default-gateway-openshift-default-fcb75c45f-2ksf9\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:27.803414 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:27.803375    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:28.134676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.134646    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"]
Oct 30 00:50:28.135317 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:50:28.135289    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d44716_de3f_4435_a9ad_00da8719aee7.slice/crio-2da3e313ec3e2da1897bf86fe565ed270dda719ad4aafbd7ff33b81105866e6c WatchSource:0}: Error finding container 2da3e313ec3e2da1897bf86fe565ed270dda719ad4aafbd7ff33b81105866e6c: Status 404 returned error can't find the container with id 2da3e313ec3e2da1897bf86fe565ed270dda719ad4aafbd7ff33b81105866e6c
Oct 30 00:50:28.137488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.137459    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:50:28.137589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.137536    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:50:28.137589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.137581    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:50:28.905830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.905792    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" event={"ID":"c5d44716-de3f-4435-a9ad-00da8719aee7","Type":"ContainerStarted","Data":"a64d05a4e28f6338120bfc31299a74245ac7073c245c58a75a48247f085eff0f"}
Oct 30 00:50:28.906035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.905861    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" event={"ID":"c5d44716-de3f-4435-a9ad-00da8719aee7","Type":"ContainerStarted","Data":"2da3e313ec3e2da1897bf86fe565ed270dda719ad4aafbd7ff33b81105866e6c"}
Oct 30 00:50:28.972165 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:28.972112    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podStartSLOduration=1.972097059 podStartE2EDuration="1.972097059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:28.97007695 +0000 UTC m=+536.999387180" watchObservedRunningTime="2026-10-30 00:50:28.972097059 +0000 UTC m=+537.001407290"
Oct 30 00:50:29.804604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:29.804563    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:29.806142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:29.806115    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 00:50:29.806276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:29.806163    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 00:50:30.804572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:30.804535    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 00:50:30.804760 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:30.804591    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 00:50:31.804760 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:31.804723    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 00:50:31.805243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:31.804778    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 00:50:32.804407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:32.804370    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 00:50:32.804594 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:32.804440    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 00:50:33.804789 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:33.804751    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 00:50:33.805225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:33.804812    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 00:50:34.804461 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:34.804418    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 00:50:34.804679 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:34.804473    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 00:50:36.289648 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:36.289614    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:36.290171 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:36.289971    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:36.290715 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:36.290691    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 00:50:48.948988 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:48.948952    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-976b87cf6-rjtkz"]
Oct 30 00:50:48.952396 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:48.952379    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:48.970107 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:48.970074    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-976b87cf6-rjtkz"]
Oct 30 00:50:48.982529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:48.982505    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2xt6n\""
Oct 30 00:50:48.982529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:48.982505    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 00:50:49.038959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.038922    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f9hz\" (UniqueName: \"kubernetes.io/projected/c86df6e2-f26b-4af0-976a-c4f26169dcb8-kube-api-access-9f9hz\") pod \"authorino-976b87cf6-rjtkz\" (UID: \"c86df6e2-f26b-4af0-976a-c4f26169dcb8\") " pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.039172 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.039089    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c86df6e2-f26b-4af0-976a-c4f26169dcb8-tls-cert\") pod \"authorino-976b87cf6-rjtkz\" (UID: \"c86df6e2-f26b-4af0-976a-c4f26169dcb8\") " pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.140223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.140186    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c86df6e2-f26b-4af0-976a-c4f26169dcb8-tls-cert\") pod \"authorino-976b87cf6-rjtkz\" (UID: \"c86df6e2-f26b-4af0-976a-c4f26169dcb8\") " pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.140394 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.140242    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f9hz\" (UniqueName: \"kubernetes.io/projected/c86df6e2-f26b-4af0-976a-c4f26169dcb8-kube-api-access-9f9hz\") pod \"authorino-976b87cf6-rjtkz\" (UID: \"c86df6e2-f26b-4af0-976a-c4f26169dcb8\") " pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.142967 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.142941    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c86df6e2-f26b-4af0-976a-c4f26169dcb8-tls-cert\") pod \"authorino-976b87cf6-rjtkz\" (UID: \"c86df6e2-f26b-4af0-976a-c4f26169dcb8\") " pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.165973 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.165940    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f9hz\" (UniqueName: \"kubernetes.io/projected/c86df6e2-f26b-4af0-976a-c4f26169dcb8-kube-api-access-9f9hz\") pod \"authorino-976b87cf6-rjtkz\" (UID: \"c86df6e2-f26b-4af0-976a-c4f26169dcb8\") " pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.276142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.276110    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-976b87cf6-rjtkz"
Oct 30 00:50:49.576681 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:49.576656    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-976b87cf6-rjtkz"]
Oct 30 00:50:49.577656 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:50:49.577627    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc86df6e2_f26b_4af0_976a_c4f26169dcb8.slice/crio-e9263045962a569c260ac45a31b7ea5b727a02cc0863e514ef5653751d2f7474 WatchSource:0}: Error finding container e9263045962a569c260ac45a31b7ea5b727a02cc0863e514ef5653751d2f7474: Status 404 returned error can't find the container with id e9263045962a569c260ac45a31b7ea5b727a02cc0863e514ef5653751d2f7474
Oct 30 00:50:50.011004 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:50.010963    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-976b87cf6-rjtkz" event={"ID":"c86df6e2-f26b-4af0-976a-c4f26169dcb8","Type":"ContainerStarted","Data":"e9263045962a569c260ac45a31b7ea5b727a02cc0863e514ef5653751d2f7474"}
Oct 30 00:50:51.019125 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:51.019080    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-976b87cf6-rjtkz" event={"ID":"c86df6e2-f26b-4af0-976a-c4f26169dcb8","Type":"ContainerStarted","Data":"36320c68e983ee8f15aad2168e051d8496fb00c155a0863a1d934ddcc6f7e5ab"}
Oct 30 00:50:51.095112 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:50:51.095062    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-976b87cf6-rjtkz" podStartSLOduration=2.700451497 podStartE2EDuration="3.095047534s" totalImagesPullingTime="394.596037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:51.09256379 +0000 UTC m=+559.121874020" watchObservedRunningTime="2026-10-30 00:50:51.095047534 +0000 UTC m=+559.124357765"
Oct 30 00:50:57.012867 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:50:57.012812    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:51:02.255407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:51:02.255376    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:51:02.371894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:51:02.371835    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:51:04.026123 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:51:04.026084    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:51:04.158566 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:51:04.158533    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:51:18.747873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:51:18.747773    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:51:27.011567 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:51:27.011529    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:51:57.012780 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:51:57.012744    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:52:01.582717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:01.582675    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:01.695353 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:01.695322    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:18.871546 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:18.871516    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:27.012218 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:52:27.012183    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:52:36.428869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:36.428805    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:36.555096 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:36.555063    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:37.187544 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:37.187508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:43.275234 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:43.275147    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:43.385265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:43.385229    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:43.591236 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:43.591152    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:44.116932 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:44.116901    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:44.357614 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:44.357578    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:46.560089 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:46.560048    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:46.678191 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:46.678158    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:47.054266 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.054238    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"]
Oct 30 00:52:47.057869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.057834    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.084425 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.084399    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bbc42836-openshift-default-dockercfg-snqtc\""
Oct 30 00:52:47.102369 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.102340    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"]
Oct 30 00:52:47.134080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.134054    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:47.145473 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145447    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145501    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145583    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145626    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145655    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145679    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145817    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjbpg\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145837    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.145903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.145881    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.246759 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.246725    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjbpg\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.246759 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.246766    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.246799    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.246834    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.246919    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.246953    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247003    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247048    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247089    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247384    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247419    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247577 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247489    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247577 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247523    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.247934 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.247915    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.249584 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.249567    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.250049 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.250032    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.274277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.274233    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.276269 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.276212    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjbpg\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg\") pod \"e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") " pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.291874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.291811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:47.371747 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.371664    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:47.452767 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.452667    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:47.682583 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.682491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:47.697770 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.697746    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"]
Oct 30 00:52:47.699465 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:52:47.699439    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea71bf5_8d41_4793_ab87_c09d6cc1bf02.slice/crio-a534532a8c48012ab5bf322e0e9f3c41bd0fd9b8edcba27047b3c2b89a5e8e86 WatchSource:0}: Error finding container a534532a8c48012ab5bf322e0e9f3c41bd0fd9b8edcba27047b3c2b89a5e8e86: Status 404 returned error can't find the container with id a534532a8c48012ab5bf322e0e9f3c41bd0fd9b8edcba27047b3c2b89a5e8e86
Oct 30 00:52:47.701541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.701514    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:47.701623 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.701574    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:47.701623 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.701609    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:47.796818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:47.796788    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:48.488080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:48.488039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:48.620378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:48.620349    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" event={"ID":"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02","Type":"ContainerStarted","Data":"a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"}
Oct 30 00:52:48.620564 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:48.620396    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" event={"ID":"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02","Type":"ContainerStarted","Data":"a534532a8c48012ab5bf322e0e9f3c41bd0fd9b8edcba27047b3c2b89a5e8e86"}
Oct 30 00:52:48.685540 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:48.685492    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podStartSLOduration=1.685477337 podStartE2EDuration="1.685477337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:48.683411809 +0000 UTC m=+676.712722036" watchObservedRunningTime="2026-10-30 00:52:48.685477337 +0000 UTC m=+676.714787568"
Oct 30 00:52:49.012993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.012958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:49.136339 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.136302    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:49.267729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.267643    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:49.353387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.353346    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"]
Oct 30 00:52:49.356986 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.356970    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.368078 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.368054    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume\") pod \"payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") " pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.368202 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.368106    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c648\" (UniqueName: \"kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648\") pod \"payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") " pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.372028 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.371987    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:49.377755 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.377679    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-aaf98f1a\""
Oct 30 00:52:49.378456 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.378430    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:49.379200 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.379176    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"]
Oct 30 00:52:49.386374 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.386352    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:49.410891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.410863    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"]
Oct 30 00:52:49.414511 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.414486    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.431449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.431426    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-aaf98f1a-dockercfg-tr8kp\""
Oct 30 00:52:49.433439 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.433414    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"]
Oct 30 00:52:49.485878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.475831    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.485878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.475946    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume\") pod \"payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") " pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.485878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.476046    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbtdk\" (UniqueName: \"kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk\") pod \"payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.485878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.476087    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c648\" (UniqueName: \"kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648\") pod \"payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") " pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.485878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.477315    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume\") pod \"payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") " pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.512895 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.512639    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c648\" (UniqueName: \"kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648\") pod \"payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") " pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.513745 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.513717    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:49.577349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.577242    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbtdk\" (UniqueName: \"kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk\") pod \"payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.577551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.577395    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.578183 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.578156    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.602706 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.602678    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbtdk\" (UniqueName: \"kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk\") pod \"payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.626003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.625979    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:49.626909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.626872    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:52:49.671317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.671298    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:49.726975 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.726944    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:49.981783 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.981750    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"]
Oct 30 00:52:49.983632 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:52:49.983441    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039ede2c_e442_4a1d_9329_a6e14e67273e.slice/crio-768ea0758a5423d02d02e8f5741a57c91b516415a0ea9a49ff4139b1ca3d6b12 WatchSource:0}: Error finding container 768ea0758a5423d02d02e8f5741a57c91b516415a0ea9a49ff4139b1ca3d6b12: Status 404 returned error can't find the container with id 768ea0758a5423d02d02e8f5741a57c91b516415a0ea9a49ff4139b1ca3d6b12
Oct 30 00:52:49.985806 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:49.985785    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:52:50.049810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:50.049786    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"]
Oct 30 00:52:50.051776 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:52:50.051749    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3762f906_d003_42ea_9ab7_6f60c99200e7.slice/crio-2a9b389a1513a096e67d21da21ed354033ee3e14980592cd25f94731cdf02ce0 WatchSource:0}: Error finding container 2a9b389a1513a096e67d21da21ed354033ee3e14980592cd25f94731cdf02ce0: Status 404 returned error can't find the container with id 2a9b389a1513a096e67d21da21ed354033ee3e14980592cd25f94731cdf02ce0
Oct 30 00:52:50.639445 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:50.639408    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8" event={"ID":"3762f906-d003-42ea-9ab7-6f60c99200e7","Type":"ContainerStarted","Data":"2a9b389a1513a096e67d21da21ed354033ee3e14980592cd25f94731cdf02ce0"}
Oct 30 00:52:50.642002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:50.641754    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc" event={"ID":"039ede2c-e442-4a1d-9329-a6e14e67273e","Type":"ContainerStarted","Data":"768ea0758a5423d02d02e8f5741a57c91b516415a0ea9a49ff4139b1ca3d6b12"}
Oct 30 00:52:50.686373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:50.686343    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:50.823047 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:50.822934    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:51.434083 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:51.434034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:52.295954 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.295909    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:52.386644 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.386612    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"]
Oct 30 00:52:52.390923 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.390902    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.408295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.408267    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"]
Oct 30 00:52:52.413798 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.413770    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:52.419285 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.419260    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bbc42836\""
Oct 30 00:52:52.513963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.511294    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzftx\" (UniqueName: \"kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx\") pod \"payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") " pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.513963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.511342    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume\") pod \"payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") " pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.612397 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.612367    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzftx\" (UniqueName: \"kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx\") pod \"payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") " pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.612530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.612410    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume\") pod \"payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") " pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.613143 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.613123    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume\") pod \"payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") " pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.637093 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.637055    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzftx\" (UniqueName: \"kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx\") pod \"payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") " pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.656730 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.656673    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8" event={"ID":"3762f906-d003-42ea-9ab7-6f60c99200e7","Type":"ContainerStarted","Data":"fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e"}
Oct 30 00:52:52.656928 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.656881    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:52:52.658568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.658542    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc" event={"ID":"039ede2c-e442-4a1d-9329-a6e14e67273e","Type":"ContainerStarted","Data":"14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0"}
Oct 30 00:52:52.658699 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.658639    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:52:52.703058 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.702996    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8" podStartSLOduration=1.265148926 podStartE2EDuration="3.702980041s" totalImagesPullingTime="2.437831115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:52.702974239 +0000 UTC m=+680.732284532" watchObservedRunningTime="2026-10-30 00:52:52.702980041 +0000 UTC m=+680.732290273"
Oct 30 00:52:52.708144 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.708114    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:52.763191 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:52.762556    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc" podStartSLOduration=1.303164695 podStartE2EDuration="3.76253618s" totalImagesPullingTime="2.459371485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:52.760231691 +0000 UTC m=+680.789541934" watchObservedRunningTime="2026-10-30 00:52:52.76253618 +0000 UTC m=+680.791846412"
Oct 30 00:52:53.009467 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:53.009440    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"]
Oct 30 00:52:53.010902 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:52:53.010874    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9226846_94b0_4579_b9b5_4e316368b608.slice/crio-8adb634dc04ec67d53652d3f67035203688fa3d117f547cf95dff5626b04b788 WatchSource:0}: Error finding container 8adb634dc04ec67d53652d3f67035203688fa3d117f547cf95dff5626b04b788: Status 404 returned error can't find the container with id 8adb634dc04ec67d53652d3f67035203688fa3d117f547cf95dff5626b04b788
Oct 30 00:52:53.301212 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:53.301122    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:53.665697 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:53.665609    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl" event={"ID":"e9226846-94b0-4579-b9b5-4e316368b608","Type":"ContainerStarted","Data":"6a473ba6477440a48d40e2bdb0a3f66cdcddc4b43f03ce80a124f4e5d261f61d"}
Oct 30 00:52:53.665697 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:53.665662    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl" event={"ID":"e9226846-94b0-4579-b9b5-4e316368b608","Type":"ContainerStarted","Data":"8adb634dc04ec67d53652d3f67035203688fa3d117f547cf95dff5626b04b788"}
Oct 30 00:52:53.665964 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:53.665723    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:52:53.712266 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:53.712221    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl" podStartSLOduration=1.712207711 podStartE2EDuration="1.712207711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:53.710163176 +0000 UTC m=+681.739473405" watchObservedRunningTime="2026-10-30 00:52:53.712207711 +0000 UTC m=+681.741517978"
Oct 30 00:52:54.982148 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:54.982118    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:55.096878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:55.095929    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:55.218370 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:55.218335    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:56.589716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:56.589678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:56.704320 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:56.704287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:56.842649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:56.842539    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:56.969500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:56.969470    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:57.012871 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:52:57.012820    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:52:57.048091 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.048064    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"]
Oct 30 00:52:57.052091 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.052071    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.080199 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.080167    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"]
Oct 30 00:52:57.090047 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.090023    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3cc211d9-openshift-default-dockercfg-97dzq\""
Oct 30 00:52:57.116231 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.116165    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:57.161154 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161118    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161162    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161193    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mw7\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161243    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161310    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161354    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161446    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161474    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.161657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.161512    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.262885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.262827    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.262906    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.262952    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.262990    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263015    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263039    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mw7\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263074    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263390 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263155    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263390 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263206    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263623    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263639    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.263866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263744    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.264018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.263993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.264531 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.264510    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.265826 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.265799    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.265967 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.265951    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.288598 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.288575    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.288779 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.288756    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mw7\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7\") pod \"e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") " pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.367491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.367412    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:57.427350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.427315    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"]
Oct 30 00:52:57.433684 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.433286    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466005 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.465975    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466014    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466039    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466152    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466185    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k4n4\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466210    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466248    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466318    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.466359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.466344    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.468142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.468118    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-b06528d1-gw-openshift-default-dockercfg-qjnlx\""
Oct 30 00:52:57.480207 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.480183    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"]
Oct 30 00:52:57.515861 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.515817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:57.567772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.567726    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.567991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.567790    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.567991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.567873    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k4n4\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.567991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.567904    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.567991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.567934    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.567991 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.567976    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568001    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568105    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568148    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568222    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568482    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568605 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568584    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.568715 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.568692    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.570019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.569988    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.571320 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.571274    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.572086 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.572058    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.597941 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.597907    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.598264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.598018    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k4n4\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4\") pod \"e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") " pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:57.647753 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.647504    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:57.699693 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.699671    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"]
Oct 30 00:52:57.701016 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:52:57.700981    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e54bbc_d694_404a_bfc4_4ead1792f4fe.slice/crio-8781626beb600f0a88e82ae9d06b76949cc77ba4c4475fbc4235ae2105f49932 WatchSource:0}: Error finding container 8781626beb600f0a88e82ae9d06b76949cc77ba4c4475fbc4235ae2105f49932: Status 404 returned error can't find the container with id 8781626beb600f0a88e82ae9d06b76949cc77ba4c4475fbc4235ae2105f49932
Oct 30 00:52:57.703508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.703476    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:57.703592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.703540    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:57.703592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.703586    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:57.753868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:57.753822    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:58.080528 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.080496    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"]
Oct 30 00:52:58.084884 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:52:58.084839    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9ae094_df32_4e1d_9c23_06bf5eb5a70b.slice/crio-5e73bec9440780088ceb2ccf5215bcb5ad7771f3d01bf48a48467f6e2e1ea3f2 WatchSource:0}: Error finding container 5e73bec9440780088ceb2ccf5215bcb5ad7771f3d01bf48a48467f6e2e1ea3f2: Status 404 returned error can't find the container with id 5e73bec9440780088ceb2ccf5215bcb5ad7771f3d01bf48a48467f6e2e1ea3f2
Oct 30 00:52:58.087815 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.087763    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:58.087914 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.087824    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:58.087914 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.087868    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:52:58.703705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.703647    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" event={"ID":"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b","Type":"ContainerStarted","Data":"0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"}
Oct 30 00:52:58.704182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.703745    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" event={"ID":"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b","Type":"ContainerStarted","Data":"5e73bec9440780088ceb2ccf5215bcb5ad7771f3d01bf48a48467f6e2e1ea3f2"}
Oct 30 00:52:58.705270 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.705247    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" event={"ID":"a9e54bbc-d694-404a-bfc4-4ead1792f4fe","Type":"ContainerStarted","Data":"e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"}
Oct 30 00:52:58.705391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.705285    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" event={"ID":"a9e54bbc-d694-404a-bfc4-4ead1792f4fe","Type":"ContainerStarted","Data":"8781626beb600f0a88e82ae9d06b76949cc77ba4c4475fbc4235ae2105f49932"}
Oct 30 00:52:58.754963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.754927    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:52:58.755659 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.755607    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podStartSLOduration=1.755590401 podStartE2EDuration="1.755590401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:58.751495443 +0000 UTC m=+686.780805683" watchObservedRunningTime="2026-10-30 00:52:58.755590401 +0000 UTC m=+686.784900635"
Oct 30 00:52:58.800891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.800824    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podStartSLOduration=1.8008084979999999 podStartE2EDuration="1.800808498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:58.799338285 +0000 UTC m=+686.828648553" watchObservedRunningTime="2026-10-30 00:52:58.800808498 +0000 UTC m=+686.830118730"
Oct 30 00:52:58.919980 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:58.919943    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:52:59.368871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:59.368219    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:59.375256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:59.375220    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:59.711131 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:59.711047    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:59.711968 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:59.711953    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:52:59.782441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:52:59.782413    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:00.469869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:00.469481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:00.585264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:00.585232    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:00.716087 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:00.716056    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:00.717039 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:00.717022    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:00.841304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:00.841278    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:01.043782 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:01.043750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:01.172087 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:01.171991    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:01.965871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:01.964381    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:02.102558 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:02.102523    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:02.483510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:02.483390    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:02.611996 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:02.611958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:02.829261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:02.829173    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:03.680872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:03.680052    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:53:03.681366 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:03.681212    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:53:04.058505 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:04.058472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:04.182464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:04.182435    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:04.676145 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:04.676117    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:53:04.707730 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:04.707700    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:04.913442 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:04.913333    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:05.023280 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.023246    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"]
Oct 30 00:53:05.027742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.027718    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.032482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.032455    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:05.047695 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.046039    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"]
Oct 30 00:53:05.047695 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.047467    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5e61cc8d\""
Oct 30 00:53:05.149634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.149305    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvr6\" (UniqueName: \"kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6\") pod \"payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") " pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.149634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.149413    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") " pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.250642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.250602    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") " pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.250806 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.250785    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvr6\" (UniqueName: \"kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6\") pod \"payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") " pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.251415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.251394    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") " pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.276080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.276053    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvr6\" (UniqueName: \"kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6\") pod \"payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") " pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.341862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.341809    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.650923 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.650817    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"]
Oct 30 00:53:05.666836 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:05.655814    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea9ccff_e087_490f_a4c7_5da3cfb051d9.slice/crio-09c13cf343baeaeb18a5285137634be72369dd5ad6049191deef52c9297f5460 WatchSource:0}: Error finding container 09c13cf343baeaeb18a5285137634be72369dd5ad6049191deef52c9297f5460: Status 404 returned error can't find the container with id 09c13cf343baeaeb18a5285137634be72369dd5ad6049191deef52c9297f5460
Oct 30 00:53:05.669018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.668986    4577 patch_prober.go:29] interesting pod/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:53:05.669151 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.669041    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:53:05.826460 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.826423    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb" event={"ID":"dea9ccff-e087-490f-a4c7-5da3cfb051d9","Type":"ContainerStarted","Data":"d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23"}
Oct 30 00:53:05.826984 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.826481    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb" event={"ID":"dea9ccff-e087-490f-a4c7-5da3cfb051d9","Type":"ContainerStarted","Data":"09c13cf343baeaeb18a5285137634be72369dd5ad6049191deef52c9297f5460"}
Oct 30 00:53:05.826984 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.826531    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:05.870141 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:05.870078    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb" podStartSLOduration=1.870064 podStartE2EDuration="1.870064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:05.869420376 +0000 UTC m=+693.898730604" watchObservedRunningTime="2026-10-30 00:53:05.870064 +0000 UTC m=+693.899374232"
Oct 30 00:53:06.562722 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:06.562681    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:06.683538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:06.683496    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:08.371791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.371759    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:08.485137 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.485106    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:08.595545 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.595509    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:08.847122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.847089    4577 generic.go:356] "Generic (PLEG): container finished" podID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerID="a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e" exitCode=0
Oct 30 00:53:08.847314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.847165    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" event={"ID":"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02","Type":"ContainerDied","Data":"a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"}
Oct 30 00:53:08.847580 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.847563    4577 scope.go:122] "RemoveContainer" containerID="a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"
Oct 30 00:53:08.849006 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.848967    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:08.849104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.849054    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:08.849104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:08.849094    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:09.876922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:09.876882    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" event={"ID":"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02","Type":"ContainerStarted","Data":"6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027"}
Oct 30 00:53:10.090317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:10.090279    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:10.206510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:10.206415    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:10.374868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:10.374262    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:53:10.374868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:10.374391    4577 patch_prober.go:29] interesting pod/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 00:53:10.374868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:10.374435    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 00:53:10.500193 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:10.500156    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:11.375876 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:11.372213    4577 patch_prober.go:29] interesting pod/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 00:53:11.375876 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:11.372290    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 00:53:12.120885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:12.113365    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:12.375460 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:12.375357    4577 patch_prober.go:29] interesting pod/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 00:53:12.375769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:12.375738    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 00:53:12.514871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:12.506076    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:13.372298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:13.372258    4577 patch_prober.go:29] interesting pod/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 00:53:13.372626 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:13.372596    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 00:53:13.482463 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:13.482421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:14.037686 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:14.037652    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:14.374927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:14.374302    4577 patch_prober.go:29] interesting pod/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 00:53:14.374927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:14.374380    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 00:53:15.604897 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:15.602269    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:53:15.604897 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:15.603297    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:53:15.609121 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:15.609061    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:53:15.931227 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:15.931150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:16.469242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.469193    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"]
Oct 30 00:53:16.479157 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.479123    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.486411 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.486369    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"]
Oct 30 00:53:16.497231 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.497180    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:53:16.497627 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.497594    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:53:16.543463 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.543424    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.543859 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.543818    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.544044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.544028    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk244\" (UniqueName: \"kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.645039 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.644997    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.657944 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.657901    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk244\" (UniqueName: \"kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.658347 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.658321    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.661903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.653731    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.687351 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.687314    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.706980 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.706945    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk244\" (UniqueName: \"kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244\") pod \"maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.823885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.823836    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:16.837773 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:16.837748    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:53:17.243217 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:17.242971    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"]
Oct 30 00:53:17.829118 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:17.829058    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:18.048090 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:18.047966    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl" event={"ID":"6446d457-c84e-4c4c-90f6-9f21ab6f40de","Type":"ContainerStarted","Data":"6a672a618ec9f851ce9eaf83962025a95dd32d9d830cbc62d02af32668c44fca"}
Oct 30 00:53:19.004908 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:19.004874    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:19.749482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:19.749442    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:21.076302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:21.076268    4577 generic.go:356] "Generic (PLEG): container finished" podID="6446d457-c84e-4c4c-90f6-9f21ab6f40de" containerID="2f04424a439af9623cb3b7e2ba45d17dcac5ceb195e8e2d7cac4abca6e606118" exitCode=0
Oct 30 00:53:21.076734 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:21.076351    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl" event={"ID":"6446d457-c84e-4c4c-90f6-9f21ab6f40de","Type":"ContainerDied","Data":"2f04424a439af9623cb3b7e2ba45d17dcac5ceb195e8e2d7cac4abca6e606118"}
Oct 30 00:53:21.505466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:21.505430    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:22.085516 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.085431    4577 generic.go:356] "Generic (PLEG): container finished" podID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerID="0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae" exitCode=0
Oct 30 00:53:22.086059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.085506    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" event={"ID":"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b","Type":"ContainerDied","Data":"0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"}
Oct 30 00:53:22.086059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.085983    4577 scope.go:122] "RemoveContainer" containerID="0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"
Oct 30 00:53:22.087556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.087527    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:22.087666 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.087548    4577 generic.go:356] "Generic (PLEG): container finished" podID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerID="e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd" exitCode=0
Oct 30 00:53:22.087666 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.087593    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:22.087666 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.087625    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:22.087666 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.087626    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" event={"ID":"a9e54bbc-d694-404a-bfc4-4ead1792f4fe","Type":"ContainerDied","Data":"e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"}
Oct 30 00:53:22.088129 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.088088    4577 scope.go:122] "RemoveContainer" containerID="e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"
Oct 30 00:53:22.089434 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.089406    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:22.089533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.089467    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:22.089533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.089506    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:22.279319 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.279048    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:22.387304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.385806    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244\" (UniqueName: \"kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244\") pod \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") "
Oct 30 00:53:22.387304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.385892    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca\") pod \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") "
Oct 30 00:53:22.387304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.386078    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca\") pod \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\" (UID: \"6446d457-c84e-4c4c-90f6-9f21ab6f40de\") "
Oct 30 00:53:22.387304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.386304    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca" pod "6446d457-c84e-4c4c-90f6-9f21ab6f40de" (UID: "6446d457-c84e-4c4c-90f6-9f21ab6f40de"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:22.387304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.386448    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:22.389441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.389398    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca" pod "6446d457-c84e-4c4c-90f6-9f21ab6f40de" (UID: "6446d457-c84e-4c4c-90f6-9f21ab6f40de"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:53:22.391283 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.391250    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244" pod "6446d457-c84e-4c4c-90f6-9f21ab6f40de" (UID: "6446d457-c84e-4c4c-90f6-9f21ab6f40de"). InnerVolumeSpecName "kube-api-access-hk244". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:22.488281 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.488234    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6446d457-c84e-4c4c-90f6-9f21ab6f40de-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:22.488489 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:22.488289    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk244\" (UniqueName: \"kubernetes.io/projected/6446d457-c84e-4c4c-90f6-9f21ab6f40de-kube-api-access-hk244\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:23.105260 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.105221    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" event={"ID":"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b","Type":"ContainerStarted","Data":"3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0"}
Oct 30 00:53:23.107705 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.107678    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" event={"ID":"a9e54bbc-d694-404a-bfc4-4ead1792f4fe","Type":"ContainerStarted","Data":"e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3"}
Oct 30 00:53:23.109028 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.109012    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"
Oct 30 00:53:23.109148 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.109013    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl" event={"ID":"6446d457-c84e-4c4c-90f6-9f21ab6f40de","Type":"ContainerDied","Data":"6a672a618ec9f851ce9eaf83962025a95dd32d9d830cbc62d02af32668c44fca"}
Oct 30 00:53:23.109203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.109147    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a672a618ec9f851ce9eaf83962025a95dd32d9d830cbc62d02af32668c44fca"
Oct 30 00:53:23.376926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.368538    4577 patch_prober.go:29] interesting pod/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 00:53:23.376926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.368611    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 00:53:23.376926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.368895    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:53:23.535657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.535625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:23.758360 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.758269    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:23.758687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.758665    4577 patch_prober.go:29] interesting pod/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 00:53:23.758880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:23.758832    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 00:53:24.373869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:24.371314    4577 patch_prober.go:29] interesting pod/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 00:53:24.373869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:24.371390    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 00:53:24.760869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:24.760298    4577 patch_prober.go:29] interesting pod/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 00:53:24.760869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:24.760360    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 00:53:25.372872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:25.368281    4577 patch_prober.go:29] interesting pod/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 00:53:25.372872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:25.368343    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 00:53:25.594559 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:25.594523    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:25.754879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:25.754311    4577 patch_prober.go:29] interesting pod/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 00:53:25.754879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:25.754371    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 00:53:26.071117 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.071024    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"]
Oct 30 00:53:26.072041 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.072014    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6446d457-c84e-4c4c-90f6-9f21ab6f40de" containerName="revoke-keys"
Oct 30 00:53:26.072203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.072189    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6446d457-c84e-4c4c-90f6-9f21ab6f40de" containerName="revoke-keys"
Oct 30 00:53:26.072432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.072418    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6446d457-c84e-4c4c-90f6-9f21ab6f40de" containerName="revoke-keys"
Oct 30 00:53:26.084450 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.084426    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.114917 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.110201    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"]
Oct 30 00:53:26.115189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.115162    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:53:26.116132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.116111    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:53:26.137273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.137244    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.137424 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.137282    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6t59\" (UniqueName: \"kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.137424 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.137319    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.238345 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.238308    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.238345 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.238345    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6t59\" (UniqueName: \"kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.238563 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.238373    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.239016 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.238997    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.243885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.240718    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.279769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.279733    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6t59\" (UniqueName: \"kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59\") pod \"maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.372409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.372317    4577 patch_prober.go:29] interesting pod/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 00:53:26.372715 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.372681    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 00:53:26.430629 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.430604    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:26.805893 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.796888    4577 patch_prober.go:29] interesting pod/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 00:53:26.805893 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.796942    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 00:53:26.818048 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:26.818003    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"]
Oct 30 00:53:26.818590 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:26.818560    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda777fb3f_30d1_4382_8cc0_29ba9a10a299.slice/crio-ce594f417ccf7695de89989af627d3ab2c920bbba0eec316b561f14a24f17a8c WatchSource:0}: Error finding container ce594f417ccf7695de89989af627d3ab2c920bbba0eec316b561f14a24f17a8c: Status 404 returned error can't find the container with id ce594f417ccf7695de89989af627d3ab2c920bbba0eec316b561f14a24f17a8c
Oct 30 00:53:27.008732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.008684    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"]
Oct 30 00:53:27.035011 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:27.034981    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:53:27.144902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.138833    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl" event={"ID":"a777fb3f-30d1-4382-8cc0-29ba9a10a299","Type":"ContainerStarted","Data":"ce594f417ccf7695de89989af627d3ab2c920bbba0eec316b561f14a24f17a8c"}
Oct 30 00:53:27.187138 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.187088    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:27.370395 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.370358    4577 patch_prober.go:29] interesting pod/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 00:53:27.370708 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.370664    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 00:53:27.380490 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.380465    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:27.641639 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.641598    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:27.766873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.757364    4577 patch_prober.go:29] interesting pod/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Oct 30 00:53:27.766873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:27.757449    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Oct 30 00:53:28.164872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.164809    4577 generic.go:356] "Generic (PLEG): container finished" podID="a777fb3f-30d1-4382-8cc0-29ba9a10a299" containerID="379777c68fd5cce0cee85fb2614168aa4c90d6a939e3efb5e91819dadf3784ca" exitCode=0
Oct 30 00:53:28.165413 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.164990    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl" event={"ID":"a777fb3f-30d1-4382-8cc0-29ba9a10a299","Type":"ContainerDied","Data":"379777c68fd5cce0cee85fb2614168aa4c90d6a939e3efb5e91819dadf3784ca"}
Oct 30 00:53:28.435391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.435300    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:53:28.436194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.436171    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:53:28.437330 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.437309    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:53:28.759124 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.759093    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:28.759604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.759587    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:28.760223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:28.760206    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:29.311265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.311237    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:29.321514 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.321485    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca\") pod \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") "
Oct 30 00:53:29.321658 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.321585    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca\") pod \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") "
Oct 30 00:53:29.321658 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.321639    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59\" (UniqueName: \"kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59\") pod \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\" (UID: \"a777fb3f-30d1-4382-8cc0-29ba9a10a299\") "
Oct 30 00:53:29.321955 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.321929    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca" pod "a777fb3f-30d1-4382-8cc0-29ba9a10a299" (UID: "a777fb3f-30d1-4382-8cc0-29ba9a10a299"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:29.323765 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.323733    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59" pod "a777fb3f-30d1-4382-8cc0-29ba9a10a299" (UID: "a777fb3f-30d1-4382-8cc0-29ba9a10a299"). InnerVolumeSpecName "kube-api-access-w6t59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:29.323892 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.323733    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca" pod "a777fb3f-30d1-4382-8cc0-29ba9a10a299" (UID: "a777fb3f-30d1-4382-8cc0-29ba9a10a299"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:53:29.422500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.422409    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6t59\" (UniqueName: \"kubernetes.io/projected/a777fb3f-30d1-4382-8cc0-29ba9a10a299-kube-api-access-w6t59\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:29.422500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.422442    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:29.422500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.422453    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a777fb3f-30d1-4382-8cc0-29ba9a10a299-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:29.665889 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:29.665757    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:30.178030 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:30.178001    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"
Oct 30 00:53:30.178215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:30.178042    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl" event={"ID":"a777fb3f-30d1-4382-8cc0-29ba9a10a299","Type":"ContainerDied","Data":"ce594f417ccf7695de89989af627d3ab2c920bbba0eec316b561f14a24f17a8c"}
Oct 30 00:53:30.178215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:30.178089    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce594f417ccf7695de89989af627d3ab2c920bbba0eec316b561f14a24f17a8c"
Oct 30 00:53:31.567746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:31.567708    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:32.009350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.009300    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl" podUID="e9226846-94b0-4579-b9b5-4e316368b608" containerName="payload-processing" containerID="cri-o://6a473ba6477440a48d40e2bdb0a3f66cdcddc4b43f03ce80a124f4e5d261f61d" gracePeriod=25
Oct 30 00:53:32.194710 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.194682    4577 generic.go:356] "Generic (PLEG): container finished" podID="e9226846-94b0-4579-b9b5-4e316368b608" containerID="6a473ba6477440a48d40e2bdb0a3f66cdcddc4b43f03ce80a124f4e5d261f61d" exitCode=0
Oct 30 00:53:32.194907 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.194717    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl" event={"ID":"e9226846-94b0-4579-b9b5-4e316368b608","Type":"ContainerDied","Data":"6a473ba6477440a48d40e2bdb0a3f66cdcddc4b43f03ce80a124f4e5d261f61d"}
Oct 30 00:53:32.275215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.275190    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:53:32.353575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.353548    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume\") pod \"e9226846-94b0-4579-b9b5-4e316368b608\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") "
Oct 30 00:53:32.353733 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.353691    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx\" (UniqueName: \"kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx\") pod \"e9226846-94b0-4579-b9b5-4e316368b608\" (UID: \"e9226846-94b0-4579-b9b5-4e316368b608\") "
Oct 30 00:53:32.353942 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.353920    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume" pod "e9226846-94b0-4579-b9b5-4e316368b608" (UID: "e9226846-94b0-4579-b9b5-4e316368b608"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:32.356053 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.356027    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx" pod "e9226846-94b0-4579-b9b5-4e316368b608" (UID: "e9226846-94b0-4579-b9b5-4e316368b608"). InnerVolumeSpecName "kube-api-access-rzftx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:32.454665 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.454633    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzftx\" (UniqueName: \"kubernetes.io/projected/e9226846-94b0-4579-b9b5-4e316368b608-kube-api-access-rzftx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:32.454665 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.454661    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9226846-94b0-4579-b9b5-4e316368b608-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:32.738096 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.738065    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"]
Oct 30 00:53:32.772473 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.772441    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"]
Oct 30 00:53:32.965643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:32.965585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:33.135258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.135137    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:33.201911 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.201875    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"
Oct 30 00:53:33.201911 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.201898    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl" event={"ID":"e9226846-94b0-4579-b9b5-4e316368b608","Type":"ContainerDied","Data":"8adb634dc04ec67d53652d3f67035203688fa3d117f547cf95dff5626b04b788"}
Oct 30 00:53:33.202147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.201955    4577 scope.go:122] "RemoveContainer" containerID="6a473ba6477440a48d40e2bdb0a3f66cdcddc4b43f03ce80a124f4e5d261f61d"
Oct 30 00:53:33.262393 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.262369    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"]
Oct 30 00:53:33.273415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.273393    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bbc42836-78994647fd-6hjrl"]
Oct 30 00:53:33.500505 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:33.500474    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:34.111175 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:34.111144    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9226846-94b0-4579-b9b5-4e316368b608" path="/var/lib/kubelet/pods/e9226846-94b0-4579-b9b5-4e316368b608/volumes"
Oct 30 00:53:35.077777 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.077745    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:35.613438 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.613404    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:35.742001 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.741972    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"]
Oct 30 00:53:35.746142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.746097    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:35.755388 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.755364    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bbc42836-fkkfl"]
Oct 30 00:53:35.983555 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.983514    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"]
Oct 30 00:53:35.983861 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:35.983811    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy" containerID="cri-o://6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027" gracePeriod=30
Oct 30 00:53:36.042984 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:36.042959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:36.118878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:36.118818    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6446d457-c84e-4c4c-90f6-9f21ab6f40de" path="/var/lib/kubelet/pods/6446d457-c84e-4c4c-90f6-9f21ab6f40de/volumes"
Oct 30 00:53:36.739716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:36.739685    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:36.866106 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:36.866073    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:36.981678 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:36.981647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:37.504619 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:37.504586    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:37.624262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:37.624226    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:37.739382 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:37.739338    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc" podUID="039ede2c-e442-4a1d-9329-a6e14e67273e" containerName="payload-processing" containerID="cri-o://14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0" gracePeriod=25
Oct 30 00:53:37.781869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:37.779134    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8" podUID="3762f906-d003-42ea-9ab7-6f60c99200e7" containerName="payload-pre-processing" containerID="cri-o://fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e" gracePeriod=25
Oct 30 00:53:38.043214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.043185    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:53:38.084521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.084499    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:53:38.116284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.116250    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648\" (UniqueName: \"kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648\") pod \"039ede2c-e442-4a1d-9329-a6e14e67273e\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") "
Oct 30 00:53:38.116449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.116353    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume\") pod \"3762f906-d003-42ea-9ab7-6f60c99200e7\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") "
Oct 30 00:53:38.116449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.116384    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk\" (UniqueName: \"kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk\") pod \"3762f906-d003-42ea-9ab7-6f60c99200e7\" (UID: \"3762f906-d003-42ea-9ab7-6f60c99200e7\") "
Oct 30 00:53:38.116570 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.116466    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume\") pod \"039ede2c-e442-4a1d-9329-a6e14e67273e\" (UID: \"039ede2c-e442-4a1d-9329-a6e14e67273e\") "
Oct 30 00:53:38.117180 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.117146    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume" pod "3762f906-d003-42ea-9ab7-6f60c99200e7" (UID: "3762f906-d003-42ea-9ab7-6f60c99200e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:38.117365 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.117293    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume" pod "039ede2c-e442-4a1d-9329-a6e14e67273e" (UID: "039ede2c-e442-4a1d-9329-a6e14e67273e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:38.118908 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.118883    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk" pod "3762f906-d003-42ea-9ab7-6f60c99200e7" (UID: "3762f906-d003-42ea-9ab7-6f60c99200e7"). InnerVolumeSpecName "kube-api-access-mbtdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:38.119247 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.119226    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648" pod "039ede2c-e442-4a1d-9329-a6e14e67273e" (UID: "039ede2c-e442-4a1d-9329-a6e14e67273e"). InnerVolumeSpecName "kube-api-access-5c648". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:38.120833 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.120811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:38.217987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.217951    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039ede2c-e442-4a1d-9329-a6e14e67273e-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:38.217987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.217985    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c648\" (UniqueName: \"kubernetes.io/projected/039ede2c-e442-4a1d-9329-a6e14e67273e-kube-api-access-5c648\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:38.217987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.217996    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3762f906-d003-42ea-9ab7-6f60c99200e7-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:38.218224 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.218004    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbtdk\" (UniqueName: \"kubernetes.io/projected/3762f906-d003-42ea-9ab7-6f60c99200e7-kube-api-access-mbtdk\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:38.264685 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.264650    4577 generic.go:356] "Generic (PLEG): container finished" podID="3762f906-d003-42ea-9ab7-6f60c99200e7" containerID="fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e" exitCode=0
Oct 30 00:53:38.264890 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.264705    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"
Oct 30 00:53:38.264890 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.264737    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8" event={"ID":"3762f906-d003-42ea-9ab7-6f60c99200e7","Type":"ContainerDied","Data":"fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e"}
Oct 30 00:53:38.264890 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.264795    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8" event={"ID":"3762f906-d003-42ea-9ab7-6f60c99200e7","Type":"ContainerDied","Data":"2a9b389a1513a096e67d21da21ed354033ee3e14980592cd25f94731cdf02ce0"}
Oct 30 00:53:38.264890 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.264817    4577 scope.go:122] "RemoveContainer" containerID="fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e"
Oct 30 00:53:38.267000 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.266632    4577 generic.go:356] "Generic (PLEG): container finished" podID="039ede2c-e442-4a1d-9329-a6e14e67273e" containerID="14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0" exitCode=0
Oct 30 00:53:38.267000 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.266713    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc" event={"ID":"039ede2c-e442-4a1d-9329-a6e14e67273e","Type":"ContainerDied","Data":"14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0"}
Oct 30 00:53:38.267000 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.266754    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc" event={"ID":"039ede2c-e442-4a1d-9329-a6e14e67273e","Type":"ContainerDied","Data":"768ea0758a5423d02d02e8f5741a57c91b516415a0ea9a49ff4139b1ca3d6b12"}
Oct 30 00:53:38.267000 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.266825    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"
Oct 30 00:53:38.276246 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.276226    4577 scope.go:122] "RemoveContainer" containerID="fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e"
Oct 30 00:53:38.276526 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:38.276507    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e\": container with ID starting with fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e not found: ID does not exist" containerID="fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e"
Oct 30 00:53:38.276591 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.276535    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e"} err="failed to get container status \"fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e\": rpc error: code = NotFound desc = could not find container \"fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e\": container with ID starting with fa468cac680edb2fc88be9987cfc022bf188d6276ae4f499d57025f16708f54e not found: ID does not exist"
Oct 30 00:53:38.276591 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.276554    4577 scope.go:122] "RemoveContainer" containerID="14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0"
Oct 30 00:53:38.285326 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.285304    4577 scope.go:122] "RemoveContainer" containerID="14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0"
Oct 30 00:53:38.285575 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:38.285556    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0\": container with ID starting with 14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0 not found: ID does not exist" containerID="14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0"
Oct 30 00:53:38.285630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.285583    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0"} err="failed to get container status \"14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0\": rpc error: code = NotFound desc = could not find container \"14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0\": container with ID starting with 14c330bd2394c80700e4763aef3b0434a8c16c6f44b8986b13f2a36fe22b25a0 not found: ID does not exist"
Oct 30 00:53:38.292326 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.292306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:38.332579 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.332540    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"]
Oct 30 00:53:38.335832 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.335810    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-aaf98f1a-fd989cdbc-vp5s8"]
Oct 30 00:53:38.381284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.381252    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"]
Oct 30 00:53:38.392698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.392668    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-aaf98f1a-745666dbdb-hlsrc"]
Oct 30 00:53:38.478503 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.478462    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:38.638424 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:38.638327    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:39.289204 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.289170    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:39.347781 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.347749    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"]
Oct 30 00:53:39.360560 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.360535    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b06528d1-47bhl"]
Oct 30 00:53:39.420165 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.420135    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:39.535102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.535071    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"]
Oct 30 00:53:39.535333 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.535311    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" containerID="cri-o://3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0" gracePeriod=30
Oct 30 00:53:39.562927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:39.562838    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:40.109751 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:40.109717    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="039ede2c-e442-4a1d-9329-a6e14e67273e" path="/var/lib/kubelet/pods/039ede2c-e442-4a1d-9329-a6e14e67273e/volumes"
Oct 30 00:53:40.110233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:40.110212    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3762f906-d003-42ea-9ab7-6f60c99200e7" path="/var/lib/kubelet/pods/3762f906-d003-42ea-9ab7-6f60c99200e7/volumes"
Oct 30 00:53:40.110659 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:40.110641    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a777fb3f-30d1-4382-8cc0-29ba9a10a299" path="/var/lib/kubelet/pods/a777fb3f-30d1-4382-8cc0-29ba9a10a299/volumes"
Oct 30 00:53:40.146608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:40.146577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:40.305387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:40.305346    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:41.228792 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.228767    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:53:41.287132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.287059    4577 generic.go:356] "Generic (PLEG): container finished" podID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerID="6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027" exitCode=0
Oct 30 00:53:41.287132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.287105    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"
Oct 30 00:53:41.287311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.287140    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" event={"ID":"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02","Type":"ContainerDied","Data":"6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027"}
Oct 30 00:53:41.287311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.287196    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw" event={"ID":"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02","Type":"ContainerDied","Data":"a534532a8c48012ab5bf322e0e9f3c41bd0fd9b8edcba27047b3c2b89a5e8e86"}
Oct 30 00:53:41.287311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.287220    4577 scope.go:122] "RemoveContainer" containerID="6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027"
Oct 30 00:53:41.296736 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.296655    4577 scope.go:122] "RemoveContainer" containerID="a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"
Oct 30 00:53:41.306743 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.306723    4577 scope.go:122] "RemoveContainer" containerID="6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027"
Oct 30 00:53:41.307155 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:41.307045    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027\": container with ID starting with 6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027 not found: ID does not exist" containerID="6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027"
Oct 30 00:53:41.307155 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.307070    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027"} err="failed to get container status \"6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027\": rpc error: code = NotFound desc = could not find container \"6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027\": container with ID starting with 6e22054824e7d49040b87d7da685aa3fce451803661b344fe6e90f0bd8df6027 not found: ID does not exist"
Oct 30 00:53:41.307155 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.307087    4577 scope.go:122] "RemoveContainer" containerID="a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"
Oct 30 00:53:41.307376 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:41.307290    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e\": container with ID starting with a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e not found: ID does not exist" containerID="a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"
Oct 30 00:53:41.307376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.307307    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e"} err="failed to get container status \"a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e\": rpc error: code = NotFound desc = could not find container \"a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e\": container with ID starting with a7d080c43abc0167959075ec2960b79c32813773dee4dadfd809fdb902bd692e not found: ID does not exist"
Oct 30 00:53:41.354155 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354123    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354174    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354215    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354311    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354401    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354430    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354456    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354464    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:41.354677 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354556    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354677 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354611    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token\") pod \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\" (UID: \"1ea71bf5-8d41-4793-ab87-c09d6cc1bf02\") "
Oct 30 00:53:41.354749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354665    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:41.354749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354704    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:41.354749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354742    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:41.355006 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.354989    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.355088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.355009    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.355088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.355017    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.355088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.355025    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.355242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.355220    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:41.356604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.356580    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 00:53:41.356813 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.356791    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "kube-api-access-sjbpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:41.357344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.357317    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:41.357421 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.357335    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy" pod "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" (UID: "1ea71bf5-8d41-4793-ab87-c09d6cc1bf02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:41.456133 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.456105    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjbpg\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-kube-api-access-sjbpg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.456133 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.456131    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.456133 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.456140    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.456324 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.456150    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.456324 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.456159    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:41.608173 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.608139    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:41.649300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.649231    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"]
Oct 30 00:53:41.651054 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:41.651014    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bbc42836-openshift-default-54bf5fcd57-44znw"]
Oct 30 00:53:42.110350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:42.110290    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" path="/var/lib/kubelet/pods/1ea71bf5-8d41-4793-ab87-c09d6cc1bf02/volumes"
Oct 30 00:53:43.683206 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:43.683170    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:43.762506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:43.762466    4577 patch_prober.go:29] interesting pod/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 30 00:53:43.762689 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:43.762542    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 00:53:43.794738 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:43.794711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:43.906990 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:43.906958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:44.806890 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.806807    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:44.899949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.899913    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.899949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.899950    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900019    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900073    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900097    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900138    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900170    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900189    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900240    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert\") pod \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\" (UID: \"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b\") "
Oct 30 00:53:44.900521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900283    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:44.900521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900510    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:44.900682 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900530    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:44.900762 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900739    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:44.900762 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900748    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:44.900957 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.900933    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:44.902976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.902949    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:53:44.903109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.903036    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 00:53:44.903346 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.903319    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:44.904134 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:44.904116    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4" pod "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" (UID: "1d9ae094-df32-4e1d-9c23-06bf5eb5a70b"). InnerVolumeSpecName "kube-api-access-7k4n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:45.001822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001788    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.001822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001815    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.001822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001824    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.001822 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001831    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.002115 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001840    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.002115 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001873    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.002115 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001880    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.002115 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.001889    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k4n4\" (UniqueName: \"kubernetes.io/projected/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b-kube-api-access-7k4n4\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:45.312971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.312937    4577 generic.go:356] "Generic (PLEG): container finished" podID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerID="3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0" exitCode=0
Oct 30 00:53:45.313174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.312994    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"
Oct 30 00:53:45.313174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.313032    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" event={"ID":"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b","Type":"ContainerDied","Data":"3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0"}
Oct 30 00:53:45.313174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.313096    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw" event={"ID":"1d9ae094-df32-4e1d-9c23-06bf5eb5a70b","Type":"ContainerDied","Data":"5e73bec9440780088ceb2ccf5215bcb5ad7771f3d01bf48a48467f6e2e1ea3f2"}
Oct 30 00:53:45.313174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.313124    4577 scope.go:122] "RemoveContainer" containerID="3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0"
Oct 30 00:53:45.323311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.323295    4577 scope.go:122] "RemoveContainer" containerID="0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"
Oct 30 00:53:45.331729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.331712    4577 scope.go:122] "RemoveContainer" containerID="3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0"
Oct 30 00:53:45.332028 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:45.332004    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0\": container with ID starting with 3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0 not found: ID does not exist" containerID="3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0"
Oct 30 00:53:45.332102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.332043    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0"} err="failed to get container status \"3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0\": rpc error: code = NotFound desc = could not find container \"3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0\": container with ID starting with 3d6911c8b069f837352a2bddd482e9e3316e4bbe1dde6b76bf83f3f791e722d0 not found: ID does not exist"
Oct 30 00:53:45.332102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.332067    4577 scope.go:122] "RemoveContainer" containerID="0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"
Oct 30 00:53:45.332351 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:45.332334    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae\": container with ID starting with 0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae not found: ID does not exist" containerID="0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"
Oct 30 00:53:45.332390 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.332358    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae"} err="failed to get container status \"0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae\": rpc error: code = NotFound desc = could not find container \"0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae\": container with ID starting with 0663a86a1c911ed18e719aff892a56757c2b8e0ae3f470c65f8be38599ad5eae not found: ID does not exist"
Oct 30 00:53:45.384244 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.384220    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"]
Oct 30 00:53:45.398114 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.398088    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-b06528d1-gw-openshift-default-68659c966f-phjbw"]
Oct 30 00:53:45.702548 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:45.702468    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:46.109617 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:46.109589    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" path="/var/lib/kubelet/pods/1d9ae094-df32-4e1d-9c23-06bf5eb5a70b/volumes"
Oct 30 00:53:47.367616 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.367568    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"]
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368088    4577 cpu_manager.go:414] "containerMap: removing container" podUID="3762f906-d003-42ea-9ab7-6f60c99200e7" containerName="payload-pre-processing"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368107    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3762f906-d003-42ea-9ab7-6f60c99200e7" containerName="payload-pre-processing"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368135    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a777fb3f-30d1-4382-8cc0-29ba9a10a299" containerName="revoke-keys"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368142    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a777fb3f-30d1-4382-8cc0-29ba9a10a299" containerName="revoke-keys"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368158    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368165    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368179    4577 cpu_manager.go:414] "containerMap: removing container" podUID="039ede2c-e442-4a1d-9329-a6e14e67273e" containerName="payload-processing"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368186    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="039ede2c-e442-4a1d-9329-a6e14e67273e" containerName="payload-processing"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368201    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e9226846-94b0-4579-b9b5-4e316368b608" containerName="payload-processing"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368207    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9226846-94b0-4579-b9b5-4e316368b608" containerName="payload-processing"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368217    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368222    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368234    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy"
Oct 30 00:53:47.368230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368240    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368246    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368251    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368337    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368347    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="039ede2c-e442-4a1d-9329-a6e14e67273e" containerName="payload-processing"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368356    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9226846-94b0-4579-b9b5-4e316368b608" containerName="payload-processing"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368366    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ea71bf5-8d41-4793-ab87-c09d6cc1bf02" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368374    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368379    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d9ae094-df32-4e1d-9c23-06bf5eb5a70b" containerName="istio-proxy"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368387    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="3762f906-d003-42ea-9ab7-6f60c99200e7" containerName="payload-pre-processing"
Oct 30 00:53:47.368902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.368395    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a777fb3f-30d1-4382-8cc0-29ba9a10a299" containerName="revoke-keys"
Oct 30 00:53:47.370820 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.370797    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.401392 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.401350    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-72ed13-openshift-default-dockercfg-tjgpb\""
Oct 30 00:53:47.404192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.404169    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"]
Oct 30 00:53:47.425260 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425234    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425294    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425443 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425426    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425451    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425514    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425531    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425625 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425603    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425655 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425635    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.425744 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.425719    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms2g6\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.434111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.434082    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:47.526472 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526438    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526624 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526485    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526624 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526601    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526761 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526641    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526761 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526677    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526761 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526718    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms2g6\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526938 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526837    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.526938 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526918    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.527056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526967    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.527056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.526997    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.527056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.527038    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.527199 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.527053    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.527280 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.527263    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.527485 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.527460    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.529127 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.529109    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.529399 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.529381    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.548916 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.548885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:47.551999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.551964    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.552234 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.552216    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms2g6\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.662469 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.662377    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:47.684024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.683987    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:47.743541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.743514    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"]
Oct 30 00:53:47.749317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.749296    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.771359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.769927    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:53:47.771359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.770115    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:53:47.781057 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.781017    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"]
Oct 30 00:53:47.833288 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.833256    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.833441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.833302    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmbc6\" (UniqueName: \"kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.833441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.833430    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.934035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.933961    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.934035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.934000    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmbc6\" (UniqueName: \"kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.934211 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.934051    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.934628 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.934602    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.936390 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.936369    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.959293 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.959266    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmbc6\" (UniqueName: \"kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6\") pod \"maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:47.999226 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:47.999198    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"]
Oct 30 00:53:48.000323 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:48.000297    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88be2916_cb08_4316_9ca4_f4a4bd0cfeca.slice/crio-1b51e8fbc2ef097d346e4ba446ec71bb5ae73ea98892784f73f8198376412883 WatchSource:0}: Error finding container 1b51e8fbc2ef097d346e4ba446ec71bb5ae73ea98892784f73f8198376412883: Status 404 returned error can't find the container with id 1b51e8fbc2ef097d346e4ba446ec71bb5ae73ea98892784f73f8198376412883
Oct 30 00:53:48.002616 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.002589    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:48.002698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.002655    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:48.002698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.002685    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:48.084728 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.084701    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:48.333051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.333002    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" event={"ID":"88be2916-cb08-4316-9ca4-f4a4bd0cfeca","Type":"ContainerStarted","Data":"b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"}
Oct 30 00:53:48.333245 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.333071    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" event={"ID":"88be2916-cb08-4316-9ca4-f4a4bd0cfeca","Type":"ContainerStarted","Data":"1b51e8fbc2ef097d346e4ba446ec71bb5ae73ea98892784f73f8198376412883"}
Oct 30 00:53:48.384551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.384490    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podStartSLOduration=1.3844733790000001 podStartE2EDuration="1.384473379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:48.382481889 +0000 UTC m=+736.411792132" watchObservedRunningTime="2026-10-30 00:53:48.384473379 +0000 UTC m=+736.413783610"
Oct 30 00:53:48.388291 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.388266    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"]
Oct 30 00:53:48.389508 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:48.389483    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46728a8a_4a16_42e5_a42d_20ac7b1a3e25.slice/crio-bc2af29bc7c7a339c50641256826083937bcd45967a596423aae0494731dccd7 WatchSource:0}: Error finding container bc2af29bc7c7a339c50641256826083937bcd45967a596423aae0494731dccd7: Status 404 returned error can't find the container with id bc2af29bc7c7a339c50641256826083937bcd45967a596423aae0494731dccd7
Oct 30 00:53:48.684103 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:48.684080    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:49.339816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:49.339781    4577 generic.go:356] "Generic (PLEG): container finished" podID="46728a8a-4a16-42e5-a42d-20ac7b1a3e25" containerID="63d0578790f70c37dc33437e2e726b885a07bbf995a93816dde7e76311ddee2e" exitCode=0
Oct 30 00:53:49.339997 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:49.339877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7" event={"ID":"46728a8a-4a16-42e5-a42d-20ac7b1a3e25","Type":"ContainerDied","Data":"63d0578790f70c37dc33437e2e726b885a07bbf995a93816dde7e76311ddee2e"}
Oct 30 00:53:49.339997 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:49.339930    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7" event={"ID":"46728a8a-4a16-42e5-a42d-20ac7b1a3e25","Type":"ContainerStarted","Data":"bc2af29bc7c7a339c50641256826083937bcd45967a596423aae0494731dccd7"}
Oct 30 00:53:49.415095 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:49.415060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:49.689367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:49.689339    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:50.346099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.346050    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:50.347537 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.347514    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:53:50.445057 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.445018    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:50.512477 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.512448    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:50.665502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.665418    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca\") pod \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") "
Oct 30 00:53:50.665502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.665475    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6\" (UniqueName: \"kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6\") pod \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") "
Oct 30 00:53:50.665706 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.665582    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca\") pod \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\" (UID: \"46728a8a-4a16-42e5-a42d-20ac7b1a3e25\") "
Oct 30 00:53:50.665993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.665970    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca" pod "46728a8a-4a16-42e5-a42d-20ac7b1a3e25" (UID: "46728a8a-4a16-42e5-a42d-20ac7b1a3e25"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:53:50.666098 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.666060    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:50.667858 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.667816    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6" pod "46728a8a-4a16-42e5-a42d-20ac7b1a3e25" (UID: "46728a8a-4a16-42e5-a42d-20ac7b1a3e25"). InnerVolumeSpecName "kube-api-access-rmbc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:53:50.668065 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.668048    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca" pod "46728a8a-4a16-42e5-a42d-20ac7b1a3e25" (UID: "46728a8a-4a16-42e5-a42d-20ac7b1a3e25"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:53:50.767592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.767558    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:50.767592 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.767586    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmbc6\" (UniqueName: \"kubernetes.io/projected/46728a8a-4a16-42e5-a42d-20ac7b1a3e25-kube-api-access-rmbc6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:53:50.850235 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.850201    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:50.998410 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:50.998358    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:51.122823 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:51.122787    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:51.352570 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:51.352500    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"
Oct 30 00:53:51.352570 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:51.352529    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7" event={"ID":"46728a8a-4a16-42e5-a42d-20ac7b1a3e25","Type":"ContainerDied","Data":"bc2af29bc7c7a339c50641256826083937bcd45967a596423aae0494731dccd7"}
Oct 30 00:53:51.352736 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:51.352581    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc2af29bc7c7a339c50641256826083937bcd45967a596423aae0494731dccd7"
Oct 30 00:53:51.358577 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:51.358549    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:52.662285 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.662247    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:52.760508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.760464    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"]
Oct 30 00:53:52.761001 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.760984    4577 cpu_manager.go:414] "containerMap: removing container" podUID="46728a8a-4a16-42e5-a42d-20ac7b1a3e25" containerName="revoke-keys"
Oct 30 00:53:52.761103 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.761003    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46728a8a-4a16-42e5-a42d-20ac7b1a3e25" containerName="revoke-keys"
Oct 30 00:53:52.761161 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.761138    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="46728a8a-4a16-42e5-a42d-20ac7b1a3e25" containerName="revoke-keys"
Oct 30 00:53:52.764533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.764501    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:52.786437 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.786411    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"]
Oct 30 00:53:52.790584 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.790560    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"]
Oct 30 00:53:52.791303 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.790653    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:52.796277 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.796255    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-72ed13\""
Oct 30 00:53:52.798276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.798251    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:52.809535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.809498    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-72ed13-dockercfg-wq8zl\""
Oct 30 00:53:52.810494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.810472    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"]
Oct 30 00:53:52.921468 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.921375    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vz2h\" (UniqueName: \"kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h\") pod \"payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:52.921830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.921798    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:52.921993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.921857    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbvjd\" (UniqueName: \"kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd\") pod \"payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:52.921993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:52.921902    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:53.022967 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.022922    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vz2h\" (UniqueName: \"kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h\") pod \"payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:53.023173 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.022974    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:53.023173 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.023005    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbvjd\" (UniqueName: \"kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd\") pod \"payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:53.023173 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.023044    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:53.023730 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.023708    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:53.023798 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.023757    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:53.048110 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.048077    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vz2h\" (UniqueName: \"kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h\") pod \"payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:53.048258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.048160    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbvjd\" (UniqueName: \"kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd\") pod \"payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:53.081508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.081477    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:53.104182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.104141    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:53.383463 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.383437    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"]
Oct 30 00:53:53.385659 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:53.385626    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41e5112_952a_4a1d_841b_be3d0900b6dd.slice/crio-51efdfcabe379073453d155e4a56a8dbc34adefc7c107f0593d63433fd45a5a6 WatchSource:0}: Error finding container 51efdfcabe379073453d155e4a56a8dbc34adefc7c107f0593d63433fd45a5a6: Status 404 returned error can't find the container with id 51efdfcabe379073453d155e4a56a8dbc34adefc7c107f0593d63433fd45a5a6
Oct 30 00:53:53.416001 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.415977    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"]
Oct 30 00:53:53.423773 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:53.423743    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc273ca8_5ef5_4d77_b4aa_483a1bd9d910.slice/crio-e1d9cd11232483751cb3e2b3f49de7f2962b25ca4cd03843c19f1d2d762ba0f6 WatchSource:0}: Error finding container e1d9cd11232483751cb3e2b3f49de7f2962b25ca4cd03843c19f1d2d762ba0f6: Status 404 returned error can't find the container with id e1d9cd11232483751cb3e2b3f49de7f2962b25ca4cd03843c19f1d2d762ba0f6
Oct 30 00:53:53.423920 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.423808    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:53.961127 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:53.961086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:54.102271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.102233    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:54.371676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.371636    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h" event={"ID":"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910","Type":"ContainerStarted","Data":"458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37"}
Oct 30 00:53:54.371872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.371696    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h" event={"ID":"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910","Type":"ContainerStarted","Data":"e1d9cd11232483751cb3e2b3f49de7f2962b25ca4cd03843c19f1d2d762ba0f6"}
Oct 30 00:53:54.371872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.371760    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:53:54.373273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.373246    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs" event={"ID":"c41e5112-952a-4a1d-841b-be3d0900b6dd","Type":"ContainerStarted","Data":"0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b"}
Oct 30 00:53:54.373404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.373288    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs" event={"ID":"c41e5112-952a-4a1d-841b-be3d0900b6dd","Type":"ContainerStarted","Data":"51efdfcabe379073453d155e4a56a8dbc34adefc7c107f0593d63433fd45a5a6"}
Oct 30 00:53:54.373404 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.373339    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:53:54.420824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.420769    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h" podStartSLOduration=2.420754954 podStartE2EDuration="2.420754954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:54.418353907 +0000 UTC m=+742.447664142" watchObservedRunningTime="2026-10-30 00:53:54.420754954 +0000 UTC m=+742.450065206"
Oct 30 00:53:54.464700 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:54.464641    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs" podStartSLOduration=2.464626633 podStartE2EDuration="2.464626633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:54.462085743 +0000 UTC m=+742.491396001" watchObservedRunningTime="2026-10-30 00:53:54.464626633 +0000 UTC m=+742.493936869"
Oct 30 00:53:55.370449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:55.370401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:55.480906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:55.480868    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:55.524715 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:55.524681    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"]
Oct 30 00:53:55.525004 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:55.524976    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" containerID="cri-o://e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3" gracePeriod=30
Oct 30 00:53:55.598942 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:55.598910    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:56.161262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.161222    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:56.299340 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.299298    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:56.435502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.435406    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"]
Oct 30 00:53:56.440906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.440880    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.460275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.460251    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.460431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.460292    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2kcf\" (UniqueName: \"kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf\") pod \"payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.474503 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.474442    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-664c2b\""
Oct 30 00:53:56.492527 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.492475    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"]
Oct 30 00:53:56.552371 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.552324    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-664c2b-dockercfg-6tvlq\""
Oct 30 00:53:56.560115 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.560086    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"]
Oct 30 00:53:56.561028 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.561000    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kcf\" (UniqueName: \"kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf\") pod \"payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.561197 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.561180    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.561931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.561915    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.564505 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.564489    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.586373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.586336    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"]
Oct 30 00:53:56.590285 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.590258    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:56.597583 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.597550    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kcf\" (UniqueName: \"kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf\") pod \"payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.662703 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.662668    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.662703 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.662704    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zvwl\" (UniqueName: \"kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl\") pod \"payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.703303 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.703212    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:56.758762 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.758715    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:56.764321 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.764295    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zvwl\" (UniqueName: \"kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl\") pod \"payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.764491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.764470    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.765309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.765287    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.791031 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.790971    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zvwl\" (UniqueName: \"kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl\") pod \"payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.877374 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.877327    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:56.938253 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.938205    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"]
Oct 30 00:53:56.949097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.946247    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.962294    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"]
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967171    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967245    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967299    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967380    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967414    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967439    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967471    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cts9k\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967520    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.969079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.967568    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:56.973901 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:56.973511    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-d5b2b8-openshift-default-dockercfg-br5n4\""
Oct 30 00:53:57.015243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.015212    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:57.017675 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:53:57.017647    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:53:57.065656 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.065633    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"]
Oct 30 00:53:57.067069 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:57.067035    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8083a038_6fca_44e5_bfc8_45af539c67dd.slice/crio-8f2463e1b4da9f40b65719a03d5bdc9ddf780f78d452633f81e6b3ac9d45e3e0 WatchSource:0}: Error finding container 8f2463e1b4da9f40b65719a03d5bdc9ddf780f78d452633f81e6b3ac9d45e3e0: Status 404 returned error can't find the container with id 8f2463e1b4da9f40b65719a03d5bdc9ddf780f78d452633f81e6b3ac9d45e3e0
Oct 30 00:53:57.068188 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068162    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068288 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068204    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068288 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068230    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068288 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068255    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cts9k\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068450 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068293    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068450 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068330    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068450 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068376    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068450 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068416    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068466    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068555    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.068994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068971    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.069085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.068978    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.069085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.069010    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.069253 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.069231    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.070738 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.070722    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.071182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.071164    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.097601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.097564    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.097739 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.097677    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cts9k\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.152392 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.152358    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:57.210535 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:53:57.210504    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312cfc2d_795b_4ff1_ba0a_83d5fcbd97e8.slice/crio-db02bea7f42c268b51bd678879554bc733e32236e24a67c43abbe42931219010 WatchSource:0}: Error finding container db02bea7f42c268b51bd678879554bc733e32236e24a67c43abbe42931219010: Status 404 returned error can't find the container with id db02bea7f42c268b51bd678879554bc733e32236e24a67c43abbe42931219010
Oct 30 00:53:57.210998 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.210973    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"]
Oct 30 00:53:57.275241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.275216    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:57.393573 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.393522    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" event={"ID":"8083a038-6fca-44e5-bfc8-45af539c67dd","Type":"ContainerStarted","Data":"38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c"}
Oct 30 00:53:57.393769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.393591    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" event={"ID":"8083a038-6fca-44e5-bfc8-45af539c67dd","Type":"ContainerStarted","Data":"8f2463e1b4da9f40b65719a03d5bdc9ddf780f78d452633f81e6b3ac9d45e3e0"}
Oct 30 00:53:57.393769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.393625    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:53:57.395431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.395406    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" event={"ID":"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8","Type":"ContainerStarted","Data":"375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314"}
Oct 30 00:53:57.395581 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.395447    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" event={"ID":"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8","Type":"ContainerStarted","Data":"db02bea7f42c268b51bd678879554bc733e32236e24a67c43abbe42931219010"}
Oct 30 00:53:57.395641 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.395626    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:53:57.439641 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.439587    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" podStartSLOduration=1.439574497 podStartE2EDuration="1.439574497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:57.438211504 +0000 UTC m=+745.467521734" watchObservedRunningTime="2026-10-30 00:53:57.439574497 +0000 UTC m=+745.468884728"
Oct 30 00:53:57.484750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.484695    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" podStartSLOduration=1.484681183 podStartE2EDuration="1.484681183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:57.482307301 +0000 UTC m=+745.511617536" watchObservedRunningTime="2026-10-30 00:53:57.484681183 +0000 UTC m=+745.513991413"
Oct 30 00:53:57.546040 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.546006    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:57.605192 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.605139    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"]
Oct 30 00:53:57.618627 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.618580    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:57.618766 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.618667    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:57.618766 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:57.618712    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:53:58.402951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.402912    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" event={"ID":"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105","Type":"ContainerStarted","Data":"d31e949ee49dc6e7b5a13b19424c9a25086408711b3062cfe889600b825089d9"}
Oct 30 00:53:58.403158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.402970    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" event={"ID":"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105","Type":"ContainerStarted","Data":"0e1002811da34baf1a7d3633e6c0db96ab917336eefb2912300a51f24fc752e1"}
Oct 30 00:53:58.438925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.438893    4577 patch_prober.go:29] interesting pod/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 00:53:58.439095 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.438955    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 00:53:58.472697 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.472647    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podStartSLOduration=2.47263338 podStartE2EDuration="2.47263338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:58.465713003 +0000 UTC m=+746.495023258" watchObservedRunningTime="2026-10-30 00:53:58.47263338 +0000 UTC m=+746.501943679"
Oct 30 00:53:58.683882 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.683756    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:58.774516 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.774475    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"]
Oct 30 00:53:58.779085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.779060    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:58.790565 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.790378    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"]
Oct 30 00:53:58.796223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.796197    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-39b241\""
Oct 30 00:53:58.796747 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.796727    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-39b241-dockercfg-zdw98\""
Oct 30 00:53:58.820198 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.820168    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:58.898871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.892449    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:58.898871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.892578    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhzv\" (UniqueName: \"kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv\") pod \"payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:58.993458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.993414    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhzv\" (UniqueName: \"kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv\") pod \"payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:58.993668 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.993539    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:58.993668 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.993567    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"]
Oct 30 00:53:58.994309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:58.994290    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:59.020644 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.020619    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhzv\" (UniqueName: \"kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv\") pod \"payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:59.098500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.098470    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:53:59.164187 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.151349    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:59.276154 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.276074    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:59.276359 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.276300    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:59.281699 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.281680    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:59.409208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.409177    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:59.410119 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.410101    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:53:59.437135 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.437106    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"]
Oct 30 00:53:59.520104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.520079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:53:59.648872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:53:59.646722    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:00.418030 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.417993    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" event={"ID":"cf82a9c9-e781-431f-a03b-b0792078523c","Type":"ContainerStarted","Data":"765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b"}
Oct 30 00:54:00.418250 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.418053    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" event={"ID":"cf82a9c9-e781-431f-a03b-b0792078523c","Type":"ContainerStarted","Data":"70b0e810064ca531eb37e1dc2df878f6afa3fec12d372bf24de9ec36ce1ee099"}
Oct 30 00:54:00.418250 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.418099    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:54:00.466869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.466575    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" podStartSLOduration=2.466555065 podStartE2EDuration="2.466555065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:00.463240336 +0000 UTC m=+748.492550568" watchObservedRunningTime="2026-10-30 00:54:00.466555065 +0000 UTC m=+748.495865298"
Oct 30 00:54:00.880800 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.880776    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:54:00.911556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911529    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911559    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911593    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911647    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911667    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911685    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911741    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.911796 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911779    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.912102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911840    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7\") pod \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\" (UID: \"a9e54bbc-d694-404a-bfc4-4ead1792f4fe\") "
Oct 30 00:54:00.912102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911904    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:54:00.912102 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.911974    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:54:00.912195 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.912095    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:54:00.912265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.912238    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:54:00.912331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.912268    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:00.912331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.912291    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:54:00.912331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.912286    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:00.912331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.912321    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:00.914228 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.914199    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:54:00.914349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.914287    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "kube-api-access-g6mw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:54:00.914565 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.914544    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:54:00.914636 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:00.914544    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo" pod "a9e54bbc-d694-404a-bfc4-4ead1792f4fe" (UID: "a9e54bbc-d694-404a-bfc4-4ead1792f4fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 00:54:01.002458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.002424    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:01.013225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.013198    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6mw7\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-kube-api-access-g6mw7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:01.013225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.013221    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:01.013225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.013231    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:01.013431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.013239    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:01.013431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.013248    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:01.013431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.013255    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e54bbc-d694-404a-bfc4-4ead1792f4fe-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:01.212205 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.211338    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:01.423188 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.423092    4577 generic.go:356] "Generic (PLEG): container finished" podID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerID="e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3" exitCode=0
Oct 30 00:54:01.423188 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.423145    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"
Oct 30 00:54:01.423376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.423182    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" event={"ID":"a9e54bbc-d694-404a-bfc4-4ead1792f4fe","Type":"ContainerDied","Data":"e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3"}
Oct 30 00:54:01.423376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.423307    4577 scope.go:122] "RemoveContainer" containerID="e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3"
Oct 30 00:54:01.423637 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.423599    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56" event={"ID":"a9e54bbc-d694-404a-bfc4-4ead1792f4fe","Type":"ContainerDied","Data":"8781626beb600f0a88e82ae9d06b76949cc77ba4c4475fbc4235ae2105f49932"}
Oct 30 00:54:01.486070 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.485165    4577 scope.go:122] "RemoveContainer" containerID="e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"
Oct 30 00:54:01.494830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.494804    4577 scope.go:122] "RemoveContainer" containerID="e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3"
Oct 30 00:54:01.495307 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:54:01.495286    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3\": container with ID starting with e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3 not found: ID does not exist" containerID="e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3"
Oct 30 00:54:01.495379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.495318    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3"} err="failed to get container status \"e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3\": rpc error: code = NotFound desc = could not find container \"e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3\": container with ID starting with e31fc78d965c9802fac8574f754eecf110cc6af282c603e1e5bb48b88bb0a9c3 not found: ID does not exist"
Oct 30 00:54:01.495379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.495338    4577 scope.go:122] "RemoveContainer" containerID="e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"
Oct 30 00:54:01.495603 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:54:01.495588    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd\": container with ID starting with e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd not found: ID does not exist" containerID="e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"
Oct 30 00:54:01.495653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.495605    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd"} err="failed to get container status \"e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd\": rpc error: code = NotFound desc = could not find container \"e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd\": container with ID starting with e1f83ff987133538a7c360844bb8cd01b9a5bd449fa0276b3673b1754bc593dd not found: ID does not exist"
Oct 30 00:54:01.522332 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.522302    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"]
Oct 30 00:54:01.530312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.530289    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3cc211d9-openshift-default-79d8bfcc49-99b56"]
Oct 30 00:54:01.534644 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:01.534624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:02.110600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.110568    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" path="/var/lib/kubelet/pods/a9e54bbc-d694-404a-bfc4-4ead1792f4fe/volumes"
Oct 30 00:54:02.685182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.685151    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:02.789516 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.789477    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"]
Oct 30 00:54:02.790003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.789988    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy"
Oct 30 00:54:02.790052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.790006    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy"
Oct 30 00:54:02.790052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.790022    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy"
Oct 30 00:54:02.790052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.790027    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy"
Oct 30 00:54:02.790147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.790106    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy"
Oct 30 00:54:02.795152 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.795135    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:02.815208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.815179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:02.827248 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.827229    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-d5b2b8\""
Oct 30 00:54:02.831624 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.831604    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"]
Oct 30 00:54:02.836107 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.836082    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:02.836200 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.836132    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njhz5\" (UniqueName: \"kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5\") pod \"payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:02.890526 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.890487    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"]
Oct 30 00:54:02.891357 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.891335    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e54bbc-d694-404a-bfc4-4ead1792f4fe" containerName="istio-proxy"
Oct 30 00:54:02.895045 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.895014    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"]
Oct 30 00:54:02.895164 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.895142    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:02.912502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.912479    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-d5b2b8-dockercfg-7zlhb\""
Oct 30 00:54:02.988924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.946790    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:02.989273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.989245    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:02.991280 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.991255    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njhz5\" (UniqueName: \"kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5\") pod \"payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:02.991584 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.991561    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxk7\" (UniqueName: \"kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7\") pod \"payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:02.991782 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:02.990274    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:03.020323 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.020291    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njhz5\" (UniqueName: \"kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5\") pod \"payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:03.093211 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.093175    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxk7\" (UniqueName: \"kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7\") pod \"payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:03.093379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.093235    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:03.094180 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.094152    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:03.107990 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.107947    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:03.119169 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.119140    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxk7\" (UniqueName: \"kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7\") pod \"payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:03.209234 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.209205    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:03.389083 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.389042    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:03.441230 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:54:03.441195    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414a1c9f_095c_459b_944a_99063ca6f593.slice/crio-048d1b0e747ae9f7cac62b734b2612751c159a6a6d6a747f6e0353bf25967e93 WatchSource:0}: Error finding container 048d1b0e747ae9f7cac62b734b2612751c159a6a6d6a747f6e0353bf25967e93: Status 404 returned error can't find the container with id 048d1b0e747ae9f7cac62b734b2612751c159a6a6d6a747f6e0353bf25967e93
Oct 30 00:54:03.441992 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.441968    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"]
Oct 30 00:54:03.514354 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.514191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:03.580584 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.578114    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"]
Oct 30 00:54:03.994215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:03.994163    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb" podUID="dea9ccff-e087-490f-a4c7-5da3cfb051d9" containerName="payload-processing" containerID="cri-o://d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23" gracePeriod=25
Oct 30 00:54:04.288918 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.288894    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:54:04.307923 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.307894    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume\") pod \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") "
Oct 30 00:54:04.308065 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.308030    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6\" (UniqueName: \"kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6\") pod \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\" (UID: \"dea9ccff-e087-490f-a4c7-5da3cfb051d9\") "
Oct 30 00:54:04.308354 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.308309    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume" pod "dea9ccff-e087-490f-a4c7-5da3cfb051d9" (UID: "dea9ccff-e087-490f-a4c7-5da3cfb051d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:54:04.310314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.310291    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6" pod "dea9ccff-e087-490f-a4c7-5da3cfb051d9" (UID: "dea9ccff-e087-490f-a4c7-5da3cfb051d9"). InnerVolumeSpecName "kube-api-access-nxvr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:54:04.410097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.410059    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxvr6\" (UniqueName: \"kubernetes.io/projected/dea9ccff-e087-490f-a4c7-5da3cfb051d9-kube-api-access-nxvr6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:04.410097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.410100    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dea9ccff-e087-490f-a4c7-5da3cfb051d9-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:54:04.446618 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.445971    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c" event={"ID":"e2be05f4-9697-411f-a706-f5326cffb0c8","Type":"ContainerStarted","Data":"d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7"}
Oct 30 00:54:04.446618 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.446070    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c" event={"ID":"e2be05f4-9697-411f-a706-f5326cffb0c8","Type":"ContainerStarted","Data":"6937ebfb52483874a0a1c6ccb7c4fa6b9d9f4d7f61b02b9ed7f8ca057387042f"}
Oct 30 00:54:04.446618 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.446106    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:04.447579 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.447551    4577 generic.go:356] "Generic (PLEG): container finished" podID="dea9ccff-e087-490f-a4c7-5da3cfb051d9" containerID="d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23" exitCode=0
Oct 30 00:54:04.447711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.447617    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"
Oct 30 00:54:04.447711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.447653    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb" event={"ID":"dea9ccff-e087-490f-a4c7-5da3cfb051d9","Type":"ContainerDied","Data":"d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23"}
Oct 30 00:54:04.447711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.447689    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb" event={"ID":"dea9ccff-e087-490f-a4c7-5da3cfb051d9","Type":"ContainerDied","Data":"09c13cf343baeaeb18a5285137634be72369dd5ad6049191deef52c9297f5460"}
Oct 30 00:54:04.447934 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.447717    4577 scope.go:122] "RemoveContainer" containerID="d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23"
Oct 30 00:54:04.449873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.449792    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf" event={"ID":"414a1c9f-095c-459b-944a-99063ca6f593","Type":"ContainerStarted","Data":"2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28"}
Oct 30 00:54:04.449970 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.449876    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf" event={"ID":"414a1c9f-095c-459b-944a-99063ca6f593","Type":"ContainerStarted","Data":"048d1b0e747ae9f7cac62b734b2612751c159a6a6d6a747f6e0353bf25967e93"}
Oct 30 00:54:04.450415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.450394    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:04.473273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.473251    4577 scope.go:122] "RemoveContainer" containerID="d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23"
Oct 30 00:54:04.474054 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:54:04.473981    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23\": container with ID starting with d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23 not found: ID does not exist" containerID="d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23"
Oct 30 00:54:04.474054 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.474023    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23"} err="failed to get container status \"d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23\": rpc error: code = NotFound desc = could not find container \"d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23\": container with ID starting with d3fb89334b033602d73f5aa09a7592d8efb5f63f7b6ac439dd2407d7ca19ca23 not found: ID does not exist"
Oct 30 00:54:04.492613 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.492567    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c" podStartSLOduration=2.492553886 podStartE2EDuration="2.492553886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:04.491399289 +0000 UTC m=+752.520709520" watchObservedRunningTime="2026-10-30 00:54:04.492553886 +0000 UTC m=+752.521864154"
Oct 30 00:54:04.532539 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.532465    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"]
Oct 30 00:54:04.534897 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.534872    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5e61cc8d-86c8948996-h4tfb"]
Oct 30 00:54:04.577910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:04.577860    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf" podStartSLOduration=2.57783321 podStartE2EDuration="2.57783321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:04.575912276 +0000 UTC m=+752.605222504" watchObservedRunningTime="2026-10-30 00:54:04.57783321 +0000 UTC m=+752.607143440"
Oct 30 00:54:05.383417 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:05.383389    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 00:54:05.383972 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:05.383454    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 00:54:05.411708 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:05.411674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:06.113179 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:06.113149    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dea9ccff-e087-490f-a4c7-5da3cfb051d9" path="/var/lib/kubelet/pods/dea9ccff-e087-490f-a4c7-5da3cfb051d9/volumes"
Oct 30 00:54:06.387063 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:06.386985    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:54:06.387412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:06.387061    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:54:06.896881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:06.896817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:07.500630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:07.500599    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:07.617583 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:07.617554    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:07.653080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:07.653036    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"]
Oct 30 00:54:07.668300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:07.668269    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5e61cc8d-87kc7"]
Oct 30 00:54:07.744002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:07.743964    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:08.111608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.111534    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46728a8a-4a16-42e5-a42d-20ac7b1a3e25" path="/var/lib/kubelet/pods/46728a8a-4a16-42e5-a42d-20ac7b1a3e25/volumes"
Oct 30 00:54:08.406224 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.406143    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 00:54:08.406224 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.406209    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 00:54:08.481369 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.481325    4577 generic.go:356] "Generic (PLEG): container finished" podID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerID="b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa" exitCode=0
Oct 30 00:54:08.481532 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.481407    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" event={"ID":"88be2916-cb08-4316-9ca4-f4a4bd0cfeca","Type":"ContainerDied","Data":"b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"}
Oct 30 00:54:08.481912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.481891    4577 scope.go:122] "RemoveContainer" containerID="b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"
Oct 30 00:54:08.483296 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.483272    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:08.483385 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.483355    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:08.483449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:08.483389    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:09.514867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:09.510034    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" event={"ID":"88be2916-cb08-4316-9ca4-f4a4bd0cfeca","Type":"ContainerStarted","Data":"3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc"}
Oct 30 00:54:09.684414 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:09.684369    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:54:09.684614 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:09.684592    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 00:54:09.684704 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:09.684650    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 00:54:10.184349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:10.184313    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:10.308828 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:10.308792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:10.684801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:10.684766    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 00:54:10.685251 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:10.684820    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 00:54:11.065734 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:11.065700    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:11.684805 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:11.684768    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 00:54:11.685245 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:11.684833    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 00:54:11.771824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:11.771789    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:12.008492 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:12.008445    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:12.432494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:12.432408    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 00:54:12.686955 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:12.686330    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 00:54:12.686955 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:12.686390    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 00:54:12.781535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:12.781505    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:13.684831 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:13.684803    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 30 00:54:13.685034 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:13.684874    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 30 00:54:14.636591 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:14.636543    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:14.688272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:14.688240    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:54:14.688653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:14.688630    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:54:14.689469 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:14.689452    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 00:54:15.067482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:15.067444    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:15.462767 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:15.462681    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 00:54:16.468147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:16.468115    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 00:54:16.936449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:16.936411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:18.331620 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:18.331586    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:19.051976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.051944    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:19.173901 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.173862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:19.290587 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.290557    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:19.828216 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.828184    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"]
Oct 30 00:54:19.828738 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.828722    4577 cpu_manager.go:414] "containerMap: removing container" podUID="dea9ccff-e087-490f-a4c7-5da3cfb051d9" containerName="payload-processing"
Oct 30 00:54:19.828802 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.828739    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dea9ccff-e087-490f-a4c7-5da3cfb051d9" containerName="payload-processing"
Oct 30 00:54:19.828873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.828855    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="dea9ccff-e087-490f-a4c7-5da3cfb051d9" containerName="payload-processing"
Oct 30 00:54:19.831272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.831252    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.864254 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.858703    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-9bfe36-openshift-default-dockercfg-4n5g9\""
Oct 30 00:54:19.886442 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.886409    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"]
Oct 30 00:54:19.897750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.897706    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:19.912226 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912200    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912232    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912261    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912398    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912433    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv6fx\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912550 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912512    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912587 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912555    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912587 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912575    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:19.912665 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:19.912604    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.014233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.014196    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.014993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.014935    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.015894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.014865    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.016321 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.016301    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.016679 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.016645    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.017304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.017285    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv6fx\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.018509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.017183    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.021879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.019092    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.022249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.021591    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.022373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.021839    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.022630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.022611    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.023151 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.023132    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.023679 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.023057    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.024212 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.023641    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.031984 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.031958    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.035495 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.033041    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:20.036313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.036291    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.045295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.045268    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv6fx\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.075897 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.075544    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.144344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.144274    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:20.486414 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.486374    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"]
Oct 30 00:54:20.486574 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:54:20.486546    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd338c3c4_97df_4b76_a32b_38af17edba0b.slice/crio-8f533b754e33f66909e7762b9e61f34b6f9c5cf82c1e909fa2eac370917eb432 WatchSource:0}: Error finding container 8f533b754e33f66909e7762b9e61f34b6f9c5cf82c1e909fa2eac370917eb432: Status 404 returned error can't find the container with id 8f533b754e33f66909e7762b9e61f34b6f9c5cf82c1e909fa2eac370917eb432
Oct 30 00:54:20.489188 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.489150    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:20.489262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.489232    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:20.489307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.489293    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:20.610533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.610500    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" event={"ID":"d338c3c4-97df-4b76-a32b-38af17edba0b","Type":"ContainerStarted","Data":"21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"}
Oct 30 00:54:20.610737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.610556    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" event={"ID":"d338c3c4-97df-4b76-a32b-38af17edba0b","Type":"ContainerStarted","Data":"8f533b754e33f66909e7762b9e61f34b6f9c5cf82c1e909fa2eac370917eb432"}
Oct 30 00:54:20.672866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:20.672407    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podStartSLOduration=1.6723765130000001 podStartE2EDuration="1.672376513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:20.672157774 +0000 UTC m=+768.701468007" watchObservedRunningTime="2026-10-30 00:54:20.672376513 +0000 UTC m=+768.701686746"
Oct 30 00:54:21.006602 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.006545    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:21.138623 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.138588    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:21.144538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.144517    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:21.161422 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.161135    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:21.618586 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.618554    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:21.619657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.619633    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:21.681699 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.681662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:21.842183 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:21.842146    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:23.170671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:23.170632    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:23.322713 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:23.322610    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:23.472493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:23.472466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:23.643999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:23.643911    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:24.340912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:24.340782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:24.543792 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:24.543754    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:25.218583 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:25.218549    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:25.382443 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:25.382412    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:25.604716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:25.604686    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:26.188694 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.188669    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:26.317673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.317640    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:26.649884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.649821    4577 generic.go:356] "Generic (PLEG): container finished" podID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerID="d31e949ee49dc6e7b5a13b19424c9a25086408711b3062cfe889600b825089d9" exitCode=0
Oct 30 00:54:26.650394 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.649899    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" event={"ID":"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105","Type":"ContainerDied","Data":"d31e949ee49dc6e7b5a13b19424c9a25086408711b3062cfe889600b825089d9"}
Oct 30 00:54:26.650467 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.650397    4577 scope.go:122] "RemoveContainer" containerID="d31e949ee49dc6e7b5a13b19424c9a25086408711b3062cfe889600b825089d9"
Oct 30 00:54:26.651712 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.651682    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:26.651804 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.651785    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:26.651887 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.651830    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:26.788477 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.788436    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:26.972898 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:26.972771    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:27.033925 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:54:27.033892    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:54:27.676376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:27.676336    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" event={"ID":"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105","Type":"ContainerStarted","Data":"2da86ce71dfacf13a0cdd4627dbdd1a75bff3a8f0de05d2b550187eb6fdbca42"}
Oct 30 00:54:27.716602 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:27.716566    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:28.276334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:28.276292    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:54:28.277386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:28.277362    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 00:54:28.277491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:28.277410    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 00:54:29.276427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:29.276387    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 00:54:29.277147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:29.277109    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 00:54:29.409498 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:29.409463    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:54:29.425427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:29.425395    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:30.276629 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:30.276589    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 00:54:30.277074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:30.276676    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 00:54:31.218956 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:31.218920    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:31.279870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:31.278827    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 00:54:31.279870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:31.278906    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 00:54:32.276149 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:32.275674    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 00:54:32.276149 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:32.275735    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 00:54:32.977828 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:32.977798    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:33.092440 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:33.092408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:33.280506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:33.280480    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:54:33.281713 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:33.281687    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 00:54:34.819462 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:34.819421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:34.937976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:34.937947    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:35.795350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:35.795309    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:36.762498 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:36.762450    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:37.889025 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:37.888991    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:38.438070 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:38.438033    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:40.222990 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:40.222954    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:42.029388 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:42.029345    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:42.169339 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:42.169306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:42.612111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:42.612079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:43.815167 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:43.815136    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:45.825608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:45.825559    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:47.948066 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:47.948030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:48.844744 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:48.844624    4577 generic.go:356] "Generic (PLEG): container finished" podID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerID="21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28" exitCode=0
Oct 30 00:54:48.844744 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:48.844678    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" event={"ID":"d338c3c4-97df-4b76-a32b-38af17edba0b","Type":"ContainerDied","Data":"21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"}
Oct 30 00:54:48.845117 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:48.845103    4577 scope.go:122] "RemoveContainer" containerID="21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"
Oct 30 00:54:48.846675 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:48.846639    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:48.846814 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:48.846727    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:48.846814 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:48.846773    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:54:49.738305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:49.738271    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:49.876907 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:49.874215    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" event={"ID":"d338c3c4-97df-4b76-a32b-38af17edba0b","Type":"ContainerStarted","Data":"286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad"}
Oct 30 00:54:50.144798 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:50.144706    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:50.170074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:50.170020    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 00:54:50.170423 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:50.170370    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 00:54:51.144707 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:51.144669    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 00:54:51.145208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:51.144737    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 00:54:51.511913 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:51.511878    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:51.619267 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:51.619241    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:52.152005 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:52.151967    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 00:54:52.152600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:52.152568    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 00:54:53.153873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:53.149198    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 00:54:53.153873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:53.149259    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 00:54:53.212777 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:53.212748    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:53.329161 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:53.329125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:53.871872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:53.865511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:54.103722 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:54.103686    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:54.149043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:54.148949    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 00:54:54.149306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:54.149275    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 00:54:54.260931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:54.260894    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:55.194767 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:55.194700    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:55.196829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:55.196804    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 00:54:55.322444 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:55.322410    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:57.013777 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:54:57.013743    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:54:57.024537 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:57.024511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:57.817349 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:57.817315    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:58.817203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:58.817175    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:54:59.732768 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:54:59.732730    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:00.511731 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:00.511693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:02.302109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:02.302046    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:04.101630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:04.101593    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:05.906726 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:05.906694    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:07.712117 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:07.712078    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:09.384818 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:09.384788    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:09.525994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:09.525961    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:11.304292 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:11.304259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:13.253197 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:13.253161    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:15.049878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.049204    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:15.490286 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.490253    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"]
Oct 30 00:55:15.496145 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.496120    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.514978 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.514948    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"]
Oct 30 00:55:15.524772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.524741    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-664c2b\"/\"openshift-service-ca.crt\""
Oct 30 00:55:15.524772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.524743    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-664c2b\"/\"kube-root-ca.crt\""
Oct 30 00:55:15.524997 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.524816    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-664c2b\"/\"default-dockercfg-p5z47\""
Oct 30 00:55:15.526773 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.526750    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-664c2b\"/\"premium-w1-664c2b-kserve-self-signed-certs\""
Oct 30 00:55:15.555642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.555605    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:15.569963 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.569927    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.570131 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.569988    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.570131 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.570108    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.570241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.570131    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.570241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.570154    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z5ph\" (UniqueName: \"kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.570316 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.570266    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671116 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671079    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671137    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671163    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671192    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z5ph\" (UniqueName: \"kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671235    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671537 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671341    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671594 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671539    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671690 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671658    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.671769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.671746    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.679422 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.676574    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.679422 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.677461    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.717569 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.717534    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z5ph\" (UniqueName: \"kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph\") pod \"premium-w1-664c2b-kserve-7bdddd88c4-5xnln\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") " pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:15.820259 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:15.820172    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:16.126261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:16.126235    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"]
Oct 30 00:55:16.127271 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:16.127244    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a6c86bb_9ba7_4602_abfb_df7fc8286ba5.slice/crio-b93a2c59a9f90302cf85930616caebf794a35d30290ce527ad7291694f297334 WatchSource:0}: Error finding container b93a2c59a9f90302cf85930616caebf794a35d30290ce527ad7291694f297334: Status 404 returned error can't find the container with id b93a2c59a9f90302cf85930616caebf794a35d30290ce527ad7291694f297334
Oct 30 00:55:16.801249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:16.801200    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:17.052915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:17.052814    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" event={"ID":"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5","Type":"ContainerStarted","Data":"f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678"}
Oct 30 00:55:17.052915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:17.052873    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" event={"ID":"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5","Type":"ContainerStarted","Data":"b93a2c59a9f90302cf85930616caebf794a35d30290ce527ad7291694f297334"}
Oct 30 00:55:17.102662 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:17.102615    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podStartSLOduration=2.10260159 podStartE2EDuration="2.10260159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:17.100816187 +0000 UTC m=+825.130126420" watchObservedRunningTime="2026-10-30 00:55:17.10260159 +0000 UTC m=+825.131911821"
Oct 30 00:55:17.820745 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:17.820710    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:18.605922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:18.605884    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:19.237525 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:19.237489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:20.392994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:20.392967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:22.127430 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:22.127396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:23.814292 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:23.814258    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:25.779429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:25.779400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:25.826885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:25.825107    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:25.857868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:25.857823    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:26.130111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:26.130025    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:55:27.014052 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:55:27.014015    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:55:27.589947 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:27.589910    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:29.323433 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:29.323381    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:31.120172 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:31.120132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:32.796215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:32.796176    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:33.252263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.252230    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"]
Oct 30 00:55:33.256282 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.256262    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.275257 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.275225    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"]
Oct 30 00:55:33.284246 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.284219    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9bfe36\"/\"openshift-service-ca.crt\""
Oct 30 00:55:33.284411 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.284257    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9bfe36\"/\"default-dockercfg-phnl6\""
Oct 30 00:55:33.289493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.289476    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9bfe36\"/\"kube-root-ca.crt\""
Oct 30 00:55:33.289493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.289481    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9bfe36\"/\"premium-w0-9bfe36-kserve-self-signed-certs\""
Oct 30 00:55:33.313996 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.313965    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:33.373092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.373058    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.373092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.373100    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.373314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.373128    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.373314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.373262    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnd5r\" (UniqueName: \"kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.373314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.373298    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.373443 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.373330    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.479879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.479049    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnd5r\" (UniqueName: \"kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.479879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.479121    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.479879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.479178    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.479879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.479259    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.479879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.479295    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.479879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.479337    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.481292 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.480472    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.481292 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.481132    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.482824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.481920    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.485676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.485651    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.486205 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.486178    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.505677 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.505607    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnd5r\" (UniqueName: \"kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r\") pod \"premium-w0-9bfe36-kserve-8db555955-lpfhx\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") " pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.571995 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.571959    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:33.878248 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:33.878216    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"]
Oct 30 00:55:33.879214 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:33.879185    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd122b33_6b60_4103_a003_a8d6cdf84f27.slice/crio-42906189cc8bb60d13881ffdd4b61be0f9d2b9b3de9b680826e81aa36d0984f9 WatchSource:0}: Error finding container 42906189cc8bb60d13881ffdd4b61be0f9d2b9b3de9b680826e81aa36d0984f9: Status 404 returned error can't find the container with id 42906189cc8bb60d13881ffdd4b61be0f9d2b9b3de9b680826e81aa36d0984f9
Oct 30 00:55:34.168910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:34.168799    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" event={"ID":"fd122b33-6b60-4103-a003-a8d6cdf84f27","Type":"ContainerStarted","Data":"2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc"}
Oct 30 00:55:34.168910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:34.168854    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" event={"ID":"fd122b33-6b60-4103-a003-a8d6cdf84f27","Type":"ContainerStarted","Data":"42906189cc8bb60d13881ffdd4b61be0f9d2b9b3de9b680826e81aa36d0984f9"}
Oct 30 00:55:34.218971 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:34.218906    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podStartSLOduration=1.218891696 podStartE2EDuration="1.218891696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:34.216914982 +0000 UTC m=+842.246225214" watchObservedRunningTime="2026-10-30 00:55:34.218891696 +0000 UTC m=+842.248201927"
Oct 30 00:55:34.572264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:34.572235    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:34.601011 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:34.600980    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:36.432878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:36.432823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:38.213263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:38.213235    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:40.014233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:40.014188    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:41.947683 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:41.947646    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"]
Oct 30 00:55:41.951671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:41.951652    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:41.986488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:41.986452    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:41.999134 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:41.999107    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:55:41.999335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:41.999182    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:55:42.002261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.002237    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"]
Oct 30 00:55:42.017059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.017027    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l79vr\" (UniqueName: \"kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.017227 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.017075    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.017227 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.017207    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.117787 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.117756    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.118045 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.118019    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l79vr\" (UniqueName: \"kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.118139 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.118060    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.118784 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.118756    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.120608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.120584    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.143918 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.143833    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l79vr\" (UniqueName: \"kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr\") pod \"maas-api-revoke-sub-20584efb-j6ks7\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.304603 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.304554    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:42.622085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:42.622055    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"]
Oct 30 00:55:42.622907 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:42.622880    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87167610_fc51_4f91_bd77_8eab85d43f6c.slice/crio-1baf8bbbbcdb22c0fdbd932b911643d185e69ab29a7e408aee23b28d591ddffc WatchSource:0}: Error finding container 1baf8bbbbcdb22c0fdbd932b911643d185e69ab29a7e408aee23b28d591ddffc: Status 404 returned error can't find the container with id 1baf8bbbbcdb22c0fdbd932b911643d185e69ab29a7e408aee23b28d591ddffc
Oct 30 00:55:43.233432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.233397    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7" event={"ID":"87167610-fc51-4f91-bd77-8eab85d43f6c","Type":"ContainerStarted","Data":"7189c4bdc56b0fec50739dcf419709cc8348b74e40dbc9e6ad1b2a4289657520"}
Oct 30 00:55:43.233878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.233463    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7" event={"ID":"87167610-fc51-4f91-bd77-8eab85d43f6c","Type":"ContainerStarted","Data":"1baf8bbbbcdb22c0fdbd932b911643d185e69ab29a7e408aee23b28d591ddffc"}
Oct 30 00:55:43.279873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.279790    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7" podStartSLOduration=2.279775126 podStartE2EDuration="2.279775126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:43.277114256 +0000 UTC m=+851.306424488" watchObservedRunningTime="2026-10-30 00:55:43.279775126 +0000 UTC m=+851.309085357"
Oct 30 00:55:43.297396 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.297354    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"]
Oct 30 00:55:43.297638 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.297616    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy" containerID="cri-o://81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a" gracePeriod=30
Oct 30 00:55:43.572901 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.572791    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:43.591021 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.590991    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:43.701533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:43.701501    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:44.241472 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:44.241440    4577 generic.go:356] "Generic (PLEG): container finished" podID="87167610-fc51-4f91-bd77-8eab85d43f6c" containerID="7189c4bdc56b0fec50739dcf419709cc8348b74e40dbc9e6ad1b2a4289657520" exitCode=0
Oct 30 00:55:44.241909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:44.241537    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7" event={"ID":"87167610-fc51-4f91-bd77-8eab85d43f6c","Type":"ContainerDied","Data":"7189c4bdc56b0fec50739dcf419709cc8348b74e40dbc9e6ad1b2a4289657520"}
Oct 30 00:55:44.253156 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:44.253133    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:55:45.435163 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.435132    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:45.455828 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.455793    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca\") pod \"87167610-fc51-4f91-bd77-8eab85d43f6c\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") "
Oct 30 00:55:45.456031 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.455945    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca\") pod \"87167610-fc51-4f91-bd77-8eab85d43f6c\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") "
Oct 30 00:55:45.456074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.456036    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr\" (UniqueName: \"kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr\") pod \"87167610-fc51-4f91-bd77-8eab85d43f6c\" (UID: \"87167610-fc51-4f91-bd77-8eab85d43f6c\") "
Oct 30 00:55:45.456193 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.456167    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca" pod "87167610-fc51-4f91-bd77-8eab85d43f6c" (UID: "87167610-fc51-4f91-bd77-8eab85d43f6c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:55:45.456506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.456479    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:45.458133 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.458104    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca" pod "87167610-fc51-4f91-bd77-8eab85d43f6c" (UID: "87167610-fc51-4f91-bd77-8eab85d43f6c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:55:45.458233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.458201    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr" pod "87167610-fc51-4f91-bd77-8eab85d43f6c" (UID: "87167610-fc51-4f91-bd77-8eab85d43f6c"). InnerVolumeSpecName "kube-api-access-l79vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:55:45.511736 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.511658    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:45.562671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.562587    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/87167610-fc51-4f91-bd77-8eab85d43f6c-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:45.562671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:45.562657    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l79vr\" (UniqueName: \"kubernetes.io/projected/87167610-fc51-4f91-bd77-8eab85d43f6c-kube-api-access-l79vr\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:46.256386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:46.256353    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7" event={"ID":"87167610-fc51-4f91-bd77-8eab85d43f6c","Type":"ContainerDied","Data":"1baf8bbbbcdb22c0fdbd932b911643d185e69ab29a7e408aee23b28d591ddffc"}
Oct 30 00:55:46.256553 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:46.256399    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1baf8bbbbcdb22c0fdbd932b911643d185e69ab29a7e408aee23b28d591ddffc"
Oct 30 00:55:46.256553 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:46.256402    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"
Oct 30 00:55:47.243248 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:47.243210    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:48.582257 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.582195    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:55:48.606176 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606136    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606310 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606200    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606310 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606231    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606310 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606302    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606528 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606328    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606588    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606682 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606653    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:55:48.606742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606677    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:55:48.606742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606666    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606876 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606782    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606938 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606870    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket\") pod \"46950bd3-baa1-4289-bc9c-0a85b6548788\" (UID: \"46950bd3-baa1-4289-bc9c-0a85b6548788\") "
Oct 30 00:55:48.606938 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.606898    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:55:48.607213 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607193    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:55:48.607494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607473    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:55:48.607634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607613    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.607702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607639    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46950bd3-baa1-4289-bc9c-0a85b6548788-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.607702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607664    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.607702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607678    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.607702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.607690    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.608807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.608784    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:55:48.609046 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.608969    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 00:55:48.609185 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.609106    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "kube-api-access-xppsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:55:48.609185 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.609122    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy" pod "46950bd3-baa1-4289-bc9c-0a85b6548788" (UID: "46950bd3-baa1-4289-bc9c-0a85b6548788"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:55:48.708737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.708706    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.708737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.708731    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.708737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.708741    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xppsj\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-kube-api-access-xppsj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:48.708974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:48.708749    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46950bd3-baa1-4289-bc9c-0a85b6548788-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:49.002320 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.002283    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:49.306903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.306785    4577 generic.go:356] "Generic (PLEG): container finished" podID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerID="81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a" exitCode=0
Oct 30 00:55:49.306903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.306881    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"
Oct 30 00:55:49.306903 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.306881    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" event={"ID":"46950bd3-baa1-4289-bc9c-0a85b6548788","Type":"ContainerDied","Data":"81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a"}
Oct 30 00:55:49.307219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.306952    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4" event={"ID":"46950bd3-baa1-4289-bc9c-0a85b6548788","Type":"ContainerDied","Data":"a3e9248f2354e2637d387a20ab87f2be1495998ededfd0c6e315e648e749295d"}
Oct 30 00:55:49.307219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.306985    4577 scope.go:122] "RemoveContainer" containerID="81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a"
Oct 30 00:55:49.317953 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.317936    4577 scope.go:122] "RemoveContainer" containerID="b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"
Oct 30 00:55:49.328571 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.328555    4577 scope.go:122] "RemoveContainer" containerID="81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a"
Oct 30 00:55:49.328930 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:55:49.328911    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a\": container with ID starting with 81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a not found: ID does not exist" containerID="81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a"
Oct 30 00:55:49.329032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.328943    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a"} err="failed to get container status \"81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a\": rpc error: code = NotFound desc = could not find container \"81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a\": container with ID starting with 81017513a07317e7f8a03649c278a83f74022915dabe95174ed79310c3a3e21a not found: ID does not exist"
Oct 30 00:55:49.329032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.328966    4577 scope.go:122] "RemoveContainer" containerID="b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"
Oct 30 00:55:49.329277 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:55:49.329249    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244\": container with ID starting with b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244 not found: ID does not exist" containerID="b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"
Oct 30 00:55:49.329315 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.329299    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244"} err="failed to get container status \"b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244\": rpc error: code = NotFound desc = could not find container \"b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244\": container with ID starting with b700f2f2d6c210f0e2a55e49abf145eef8b96ce57ae5af7a8b10a1eae9090244 not found: ID does not exist"
Oct 30 00:55:49.366877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.366830    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"]
Oct 30 00:55:49.380472 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:49.380446    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ljrb4"]
Oct 30 00:55:50.110684 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.110647    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" path="/var/lib/kubelet/pods/46950bd3-baa1-4289-bc9c-0a85b6548788/volumes"
Oct 30 00:55:50.626305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626272    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"]
Oct 30 00:55:50.626766 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626752    4577 cpu_manager.go:414] "containerMap: removing container" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy"
Oct 30 00:55:50.626810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626771    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy"
Oct 30 00:55:50.626810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626786    4577 cpu_manager.go:414] "containerMap: removing container" podUID="87167610-fc51-4f91-bd77-8eab85d43f6c" containerName="revoke-keys"
Oct 30 00:55:50.626810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626792    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87167610-fc51-4f91-bd77-8eab85d43f6c" containerName="revoke-keys"
Oct 30 00:55:50.626951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626812    4577 cpu_manager.go:414] "containerMap: removing container" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy"
Oct 30 00:55:50.626951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626817    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy"
Oct 30 00:55:50.626951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626911    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy"
Oct 30 00:55:50.626951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626925    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="46950bd3-baa1-4289-bc9c-0a85b6548788" containerName="istio-proxy"
Oct 30 00:55:50.626951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.626936    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="87167610-fc51-4f91-bd77-8eab85d43f6c" containerName="revoke-keys"
Oct 30 00:55:50.631148 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.631132    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.648043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.648015    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"]
Oct 30 00:55:50.658445 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.658411    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:55:50.658604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.658590    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:55:50.731661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.731628    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.731661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.731662    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.731889 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.731746    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h6fk\" (UniqueName: \"kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.809921 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.809870    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:50.832532 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.832501    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h6fk\" (UniqueName: \"kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.832689 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.832602    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.832689 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.832621    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.833283 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.833256    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.835107 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.835085    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.869785 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.867558    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h6fk\" (UniqueName: \"kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk\") pod \"maas-api-revoke-sub-dbcf7442-6fg2p\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:50.943986 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:50.943909    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:51.245594 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.245567    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"]
Oct 30 00:55:51.246881 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:51.246811    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9902b50_2d2f_454f_b663_0d1e443c3632.slice/crio-b2747f2b8b320ff7eb5c6b5f9f2db0dcf2bc5486c77db02566ab0aacb0f16cfe WatchSource:0}: Error finding container b2747f2b8b320ff7eb5c6b5f9f2db0dcf2bc5486c77db02566ab0aacb0f16cfe: Status 404 returned error can't find the container with id b2747f2b8b320ff7eb5c6b5f9f2db0dcf2bc5486c77db02566ab0aacb0f16cfe
Oct 30 00:55:51.283078 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.283050    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"]
Oct 30 00:55:51.288035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.288017    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.304747 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.304715    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-39b241-openshift-default-dockercfg-bchvq\""
Oct 30 00:55:51.322128 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.322094    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"]
Oct 30 00:55:51.324237 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.324207    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p" event={"ID":"e9902b50-2d2f-454f-b663-0d1e443c3632","Type":"ContainerStarted","Data":"b2747f2b8b320ff7eb5c6b5f9f2db0dcf2bc5486c77db02566ab0aacb0f16cfe"}
Oct 30 00:55:51.338004 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.337974    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338131 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338023    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338148    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc49s\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338215 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338197    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338273    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338388 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338356    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338409    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338445    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.338551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.338533    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439457    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc49s\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439504    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439525    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439544    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439577    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439595    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.439687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439620    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439689    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439723    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.439993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.440038    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.440327    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.440427    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.440510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.440452    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.442241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.442217    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.442305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.442284    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.464295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.464268    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.464553 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.464535    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc49s\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.605569 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.605539    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:51.930265 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.930236    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"]
Oct 30 00:55:51.931337 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:51.931307    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2deeef02_5405_4d0c_baa7_4e6b9f9502fa.slice/crio-243e3a70491347643859515b95273b9884f24fd432b850e0ebc20802b0376afd WatchSource:0}: Error finding container 243e3a70491347643859515b95273b9884f24fd432b850e0ebc20802b0376afd: Status 404 returned error can't find the container with id 243e3a70491347643859515b95273b9884f24fd432b850e0ebc20802b0376afd
Oct 30 00:55:51.934051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.934015    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:55:51.934118 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.934101    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:55:51.934169 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:51.934153    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:55:52.331256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.331219    4577 generic.go:356] "Generic (PLEG): container finished" podID="e9902b50-2d2f-454f-b663-0d1e443c3632" containerID="bed34299f49a3ba7a426cf380d5b8bffe5b158ec46e324e13242f1931ba133e1" exitCode=0
Oct 30 00:55:52.331736 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.331306    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p" event={"ID":"e9902b50-2d2f-454f-b663-0d1e443c3632","Type":"ContainerDied","Data":"bed34299f49a3ba7a426cf380d5b8bffe5b158ec46e324e13242f1931ba133e1"}
Oct 30 00:55:52.333218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.333194    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" event={"ID":"2deeef02-5405-4d0c-baa7-4e6b9f9502fa","Type":"ContainerStarted","Data":"4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093"}
Oct 30 00:55:52.333331 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.333239    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" event={"ID":"2deeef02-5405-4d0c-baa7-4e6b9f9502fa","Type":"ContainerStarted","Data":"243e3a70491347643859515b95273b9884f24fd432b850e0ebc20802b0376afd"}
Oct 30 00:55:52.427904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.427855    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podStartSLOduration=1.427829486 podStartE2EDuration="1.427829486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:52.42534529 +0000 UTC m=+860.454655523" watchObservedRunningTime="2026-10-30 00:55:52.427829486 +0000 UTC m=+860.457139717"
Oct 30 00:55:52.606147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.606067    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:52.607713 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.607688    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:55:52.607891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.607735    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:55:52.618904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:52.618878    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:53.608412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.606447    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:55:53.608412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.606504    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:55:53.632562 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.632540    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:53.704877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.704822    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca\") pod \"e9902b50-2d2f-454f-b663-0d1e443c3632\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") "
Oct 30 00:55:53.705069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.704906    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk\" (UniqueName: \"kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk\") pod \"e9902b50-2d2f-454f-b663-0d1e443c3632\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") "
Oct 30 00:55:53.705069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.705000    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca\") pod \"e9902b50-2d2f-454f-b663-0d1e443c3632\" (UID: \"e9902b50-2d2f-454f-b663-0d1e443c3632\") "
Oct 30 00:55:53.705254 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.705230    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca" pod "e9902b50-2d2f-454f-b663-0d1e443c3632" (UID: "e9902b50-2d2f-454f-b663-0d1e443c3632"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:55:53.705442 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.705425    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:53.714893 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.711290    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca" pod "e9902b50-2d2f-454f-b663-0d1e443c3632" (UID: "e9902b50-2d2f-454f-b663-0d1e443c3632"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:55:53.714893 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.711318    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk" pod "e9902b50-2d2f-454f-b663-0d1e443c3632" (UID: "e9902b50-2d2f-454f-b663-0d1e443c3632"). InnerVolumeSpecName "kube-api-access-9h6fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:55:53.806598 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.806567    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e9902b50-2d2f-454f-b663-0d1e443c3632-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:53.806598 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:53.806597    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h6fk\" (UniqueName: \"kubernetes.io/projected/e9902b50-2d2f-454f-b663-0d1e443c3632-kube-api-access-9h6fk\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:55:54.365089 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:54.365058    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"
Oct 30 00:55:54.370204 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:54.370115    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p" event={"ID":"e9902b50-2d2f-454f-b663-0d1e443c3632","Type":"ContainerDied","Data":"b2747f2b8b320ff7eb5c6b5f9f2db0dcf2bc5486c77db02566ab0aacb0f16cfe"}
Oct 30 00:55:54.370408 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:54.370387    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2747f2b8b320ff7eb5c6b5f9f2db0dcf2bc5486c77db02566ab0aacb0f16cfe"
Oct 30 00:55:54.575867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:54.570100    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:54.612444 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:54.612379    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:55:54.613051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:54.612467    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:55:55.466867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.463990    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"]
Oct 30 00:55:55.466867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.464756    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e9902b50-2d2f-454f-b663-0d1e443c3632" containerName="revoke-keys"
Oct 30 00:55:55.466867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.464776    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9902b50-2d2f-454f-b663-0d1e443c3632" containerName="revoke-keys"
Oct 30 00:55:55.466867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.464931    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9902b50-2d2f-454f-b663-0d1e443c3632" containerName="revoke-keys"
Oct 30 00:55:55.472868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.472091    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.486264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.486235    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"]
Oct 30 00:55:55.495909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.495837    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-664c2b-openshift-default-dockercfg-vvttz\""
Oct 30 00:55:55.523421 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523390    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.523573 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523458    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.523651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523570    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.523651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523636    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.523735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523665    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfmf\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.523735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523694    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.523857 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.523824    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.524111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.524086    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.524933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.524909    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.606596 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.606553    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:55:55.606789 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.606668    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:55:55.626060 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626035    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626447 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626432    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626512    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626585 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626577    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626662    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626740    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626834 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626826    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.626936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626578    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.627020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.627007    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfmf\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.627095 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.627085    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.627196 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.627171    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.627302 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.626434    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.627367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.627281    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.627432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.627416    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.629450 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.629427    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.629738 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.629721    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.662864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.660524    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfmf\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.668862 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.668247    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:55.798356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:55.798314    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:56.170921 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.168408    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"]
Oct 30 00:55:56.189125 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:56.189096    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1acb033b_0c26_4221_bbd4_c927dd1f9bd5.slice/crio-fe985bbc3646268a2412303b4b64f8f9bc4838fcc4767776711037c8d7f7e2fb WatchSource:0}: Error finding container fe985bbc3646268a2412303b4b64f8f9bc4838fcc4767776711037c8d7f7e2fb: Status 404 returned error can't find the container with id fe985bbc3646268a2412303b4b64f8f9bc4838fcc4767776711037c8d7f7e2fb
Oct 30 00:55:56.192575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.192547    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:55:56.192637 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.192616    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:55:56.192712 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.192647    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:55:56.217413 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.217388    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:56.397182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.396799    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" event={"ID":"1acb033b-0c26-4221-bbd4-c927dd1f9bd5","Type":"ContainerStarted","Data":"d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8"}
Oct 30 00:55:56.397182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.396896    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" event={"ID":"1acb033b-0c26-4221-bbd4-c927dd1f9bd5","Type":"ContainerStarted","Data":"fe985bbc3646268a2412303b4b64f8f9bc4838fcc4767776711037c8d7f7e2fb"}
Oct 30 00:55:56.609225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.609191    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:55:56.609516 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.609481    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:55:56.799139 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:56.799104    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:55:57.044177 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:55:57.044134    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:55:57.606120 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.606081    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:55:57.606339 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.606141    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:55:57.666872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.666103    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podStartSLOduration=2.666081034 podStartE2EDuration="2.666081034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:56.486716979 +0000 UTC m=+864.516027211" watchObservedRunningTime="2026-10-30 00:55:57.666081034 +0000 UTC m=+865.695391265"
Oct 30 00:55:57.667086 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.667007    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"]
Oct 30 00:55:57.675956 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.675230    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.701466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.700122    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:55:57.701466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.700812    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:55:57.701466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.701296    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"]
Oct 30 00:55:57.782872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.777482    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.782872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.777652    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.782872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.777684    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28lnp\" (UniqueName: \"kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.801150 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.801116    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:55:57.801580 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.801167    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:55:57.879927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.879593    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.900508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.880152    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.900508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.880718    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28lnp\" (UniqueName: \"kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.900508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.881948    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.900508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.899708    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:57.906892 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:57.906829    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28lnp\" (UniqueName: \"kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp\") pod \"maas-api-revoke-sub-9b1077b0-vzddz\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:58.002451 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:58.002416    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:55:58.018734 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:58.018699    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:55:58.399882 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:55:58.397981    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a0b9f2_18d1_47a7_befa_2a6a6d918c78.slice/crio-f575d2aa8e311432e18709588ac9ffcd207e46b4783f302ee4ed1fd742d2b6aa WatchSource:0}: Error finding container f575d2aa8e311432e18709588ac9ffcd207e46b4783f302ee4ed1fd742d2b6aa: Status 404 returned error can't find the container with id f575d2aa8e311432e18709588ac9ffcd207e46b4783f302ee4ed1fd742d2b6aa
Oct 30 00:55:58.399882 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:58.398557    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"]
Oct 30 00:55:58.432052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:58.432019    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz" event={"ID":"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78","Type":"ContainerStarted","Data":"f575d2aa8e311432e18709588ac9ffcd207e46b4783f302ee4ed1fd742d2b6aa"}
Oct 30 00:55:58.799049 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:58.799005    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:55:58.799223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:58.799061    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:55:59.440413 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.440374    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz" event={"ID":"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78","Type":"ContainerStarted","Data":"9f9638937ce0cf46f88926260ea400ca9d8f354e6355b6752dd3621bcb2825cc"}
Oct 30 00:55:59.487871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.487276    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz" podStartSLOduration=2.487256175 podStartE2EDuration="2.487256175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:59.486244014 +0000 UTC m=+867.515554246" watchObservedRunningTime="2026-10-30 00:55:59.487256175 +0000 UTC m=+867.516566407"
Oct 30 00:55:59.649064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.649028    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:55:59.649350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.649320    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:55:59.751189 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.751160    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:59.752720 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.752697    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:59.776072 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.776046    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 00:55:59.799263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.799226    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:55:59.799452 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.799279    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:55:59.816261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:55:59.816200    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:00.450871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:00.450479    4577 generic.go:356] "Generic (PLEG): container finished" podID="a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" containerID="9f9638937ce0cf46f88926260ea400ca9d8f354e6355b6752dd3621bcb2825cc" exitCode=0
Oct 30 00:56:00.454870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:00.451613    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz" event={"ID":"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78","Type":"ContainerDied","Data":"9f9638937ce0cf46f88926260ea400ca9d8f354e6355b6752dd3621bcb2825cc"}
Oct 30 00:56:00.799141 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:00.799108    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:56:00.799395 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:00.799369    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:56:01.609714 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.609682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:01.636139 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.636113    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:56:01.737448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.737413    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca\") pod \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") "
Oct 30 00:56:01.737646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.737489    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca\") pod \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") "
Oct 30 00:56:01.737646 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.737538    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp\" (UniqueName: \"kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp\") pod \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\" (UID: \"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78\") "
Oct 30 00:56:01.738573 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.738540    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca" pod "a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" (UID: "a2a0b9f2-18d1-47a7-befa-2a6a6d918c78"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:56:01.740900 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.740874    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca" pod "a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" (UID: "a2a0b9f2-18d1-47a7-befa-2a6a6d918c78"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:56:01.741127 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.741093    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp" pod "a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" (UID: "a2a0b9f2-18d1-47a7-befa-2a6a6d918c78"). InnerVolumeSpecName "kube-api-access-28lnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:56:01.798798 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.798758    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:56:01.798996 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.798815    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:56:01.838702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.838672    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:01.838702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.838701    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:01.838702 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:01.838711    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28lnp\" (UniqueName: \"kubernetes.io/projected/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78-kube-api-access-28lnp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:02.481609 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:02.481582    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"
Oct 30 00:56:02.481791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:02.481638    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz" event={"ID":"a2a0b9f2-18d1-47a7-befa-2a6a6d918c78","Type":"ContainerDied","Data":"f575d2aa8e311432e18709588ac9ffcd207e46b4783f302ee4ed1fd742d2b6aa"}
Oct 30 00:56:02.481791 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:02.481708    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f575d2aa8e311432e18709588ac9ffcd207e46b4783f302ee4ed1fd742d2b6aa"
Oct 30 00:56:02.799122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:02.799033    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:56:02.799122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:02.799101    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:56:03.313190 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:03.313161    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:03.803243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:03.803215    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:56:03.803642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:03.803610    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:56:03.804364 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:03.804339    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 00:56:04.001714 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:04.001684    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:04.129756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:04.129665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:05.015720 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:05.015687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:06.920428 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:06.920380    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:08.698968 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:08.698931    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:10.410027 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:10.409995    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:12.205524 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:12.205493    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:13.919284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:13.919238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:15.063433 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.063401    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"]
Oct 30 00:56:15.063940 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.063923    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" containerName="revoke-keys"
Oct 30 00:56:15.064018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.063942    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" containerName="revoke-keys"
Oct 30 00:56:15.064073 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.064019    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" containerName="revoke-keys"
Oct 30 00:56:15.068485 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.068467    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.089497 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.089463    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"]
Oct 30 00:56:15.096097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.096071    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:56:15.098494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.098467    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:56:15.139083 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.139051    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.139241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.139169    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdtr2\" (UniqueName: \"kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.139241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.139217    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.240002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.239967    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.240002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.240002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdtr2\" (UniqueName: \"kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.240214 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.240050    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.240729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.240712    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.242526 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.242511    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.265610 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.265579    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdtr2\" (UniqueName: \"kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2\") pod \"maas-api-revoke-sub-ddddd62b-cnp5g\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.381769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.381687    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:15.682580 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.682551    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"]
Oct 30 00:56:15.684306 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:56:15.684274    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09c1d10_4db7_497f_b101_1242c8b02fa2.slice/crio-6ea629f8d76bfab60c8fe18dd37876da1d4964db8ee76f65fbc3ea2c19a78557 WatchSource:0}: Error finding container 6ea629f8d76bfab60c8fe18dd37876da1d4964db8ee76f65fbc3ea2c19a78557: Status 404 returned error can't find the container with id 6ea629f8d76bfab60c8fe18dd37876da1d4964db8ee76f65fbc3ea2c19a78557
Oct 30 00:56:15.710099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:15.710048    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:16.588194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:16.587831    4577 generic.go:356] "Generic (PLEG): container finished" podID="b09c1d10-4db7-497f-b101-1242c8b02fa2" containerID="4c83d4f0834aa6fe9147decf6c9c9da4b9f0682081c5e085922411ff516c1905" exitCode=0
Oct 30 00:56:16.588194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:16.587876    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g" event={"ID":"b09c1d10-4db7-497f-b101-1242c8b02fa2","Type":"ContainerDied","Data":"4c83d4f0834aa6fe9147decf6c9c9da4b9f0682081c5e085922411ff516c1905"}
Oct 30 00:56:16.588194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:16.587971    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g" event={"ID":"b09c1d10-4db7-497f-b101-1242c8b02fa2","Type":"ContainerStarted","Data":"6ea629f8d76bfab60c8fe18dd37876da1d4964db8ee76f65fbc3ea2c19a78557"}
Oct 30 00:56:17.606945 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.606908    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:17.802429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.802401    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:17.889854 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.889756    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca\") pod \"b09c1d10-4db7-497f-b101-1242c8b02fa2\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") "
Oct 30 00:56:17.889854 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.889814    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca\") pod \"b09c1d10-4db7-497f-b101-1242c8b02fa2\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") "
Oct 30 00:56:17.889854 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.889837    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2\" (UniqueName: \"kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2\") pod \"b09c1d10-4db7-497f-b101-1242c8b02fa2\" (UID: \"b09c1d10-4db7-497f-b101-1242c8b02fa2\") "
Oct 30 00:56:17.890247 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.890210    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca" pod "b09c1d10-4db7-497f-b101-1242c8b02fa2" (UID: "b09c1d10-4db7-497f-b101-1242c8b02fa2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:56:17.892251 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.892220    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2" pod "b09c1d10-4db7-497f-b101-1242c8b02fa2" (UID: "b09c1d10-4db7-497f-b101-1242c8b02fa2"). InnerVolumeSpecName "kube-api-access-cdtr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:56:17.892251 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.892233    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca" pod "b09c1d10-4db7-497f-b101-1242c8b02fa2" (UID: "b09c1d10-4db7-497f-b101-1242c8b02fa2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:56:17.991386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.991349    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdtr2\" (UniqueName: \"kubernetes.io/projected/b09c1d10-4db7-497f-b101-1242c8b02fa2-kube-api-access-cdtr2\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:17.991386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.991378    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:17.991386 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:17.991387    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b09c1d10-4db7-497f-b101-1242c8b02fa2-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:18.601438 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:18.601409    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"
Oct 30 00:56:18.601438 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:18.601410    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g" event={"ID":"b09c1d10-4db7-497f-b101-1242c8b02fa2","Type":"ContainerDied","Data":"6ea629f8d76bfab60c8fe18dd37876da1d4964db8ee76f65fbc3ea2c19a78557"}
Oct 30 00:56:18.601653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:18.601475    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ea629f8d76bfab60c8fe18dd37876da1d4964db8ee76f65fbc3ea2c19a78557"
Oct 30 00:56:19.355272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:19.355238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:19.474391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:19.473712    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:21.347530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:21.347494    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:22.391558 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.391520    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"]
Oct 30 00:56:22.392147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.392078    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b09c1d10-4db7-497f-b101-1242c8b02fa2" containerName="revoke-keys"
Oct 30 00:56:22.392147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.392092    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b09c1d10-4db7-497f-b101-1242c8b02fa2" containerName="revoke-keys"
Oct 30 00:56:22.392273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.392171    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b09c1d10-4db7-497f-b101-1242c8b02fa2" containerName="revoke-keys"
Oct 30 00:56:22.395629 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.395608    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.412387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.412360    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"]
Oct 30 00:56:22.421407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.421386    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:56:22.421747 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.421731    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:56:22.541470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.541433    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.541679 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.541489    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnqgt\" (UniqueName: \"kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.541763 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.541689    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.643197 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.643116    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.643197 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.643161    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqgt\" (UniqueName: \"kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.643393 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.643230    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.643882 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.643831    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.645693 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.645663    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.668821 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.668793    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnqgt\" (UniqueName: \"kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt\") pod \"maas-api-revoke-sub-ff0379f8-lxrlc\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:22.709868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:22.709810    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:23.012857 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:23.012821    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"]
Oct 30 00:56:23.014602 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:56:23.014570    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd748a7_8da3_44d6_b26c_d3f3eb034c99.slice/crio-b5bf6ad98fb13eb0a790a4d3efcdf27a1d9dfb34599d7221029d8569e5a63b91 WatchSource:0}: Error finding container b5bf6ad98fb13eb0a790a4d3efcdf27a1d9dfb34599d7221029d8569e5a63b91: Status 404 returned error can't find the container with id b5bf6ad98fb13eb0a790a4d3efcdf27a1d9dfb34599d7221029d8569e5a63b91
Oct 30 00:56:23.251413 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:23.251386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:23.644036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:23.643994    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc" event={"ID":"0bd748a7-8da3-44d6-b26c-d3f3eb034c99","Type":"ContainerStarted","Data":"b9be24b9686f1e178c04e943f63b2e406bad33f0d1b53a6e566632d7cc4f96c8"}
Oct 30 00:56:23.644500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:23.644064    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc" event={"ID":"0bd748a7-8da3-44d6-b26c-d3f3eb034c99","Type":"ContainerStarted","Data":"b5bf6ad98fb13eb0a790a4d3efcdf27a1d9dfb34599d7221029d8569e5a63b91"}
Oct 30 00:56:23.690982 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:23.690734    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc" podStartSLOduration=1.690714642 podStartE2EDuration="1.690714642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:56:23.687737852 +0000 UTC m=+891.717048084" watchObservedRunningTime="2026-10-30 00:56:23.690714642 +0000 UTC m=+891.720024876"
Oct 30 00:56:24.653474 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:24.653430    4577 generic.go:356] "Generic (PLEG): container finished" podID="0bd748a7-8da3-44d6-b26c-d3f3eb034c99" containerID="b9be24b9686f1e178c04e943f63b2e406bad33f0d1b53a6e566632d7cc4f96c8" exitCode=0
Oct 30 00:56:24.653948 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:24.653522    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc" event={"ID":"0bd748a7-8da3-44d6-b26c-d3f3eb034c99","Type":"ContainerDied","Data":"b9be24b9686f1e178c04e943f63b2e406bad33f0d1b53a6e566632d7cc4f96c8"}
Oct 30 00:56:25.123481 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.123443    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:25.813745 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.813720    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:25.894452 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.894413    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca\") pod \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") "
Oct 30 00:56:25.894649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.894565    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca\") pod \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") "
Oct 30 00:56:25.894649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.894636    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt\" (UniqueName: \"kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt\") pod \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\" (UID: \"0bd748a7-8da3-44d6-b26c-d3f3eb034c99\") "
Oct 30 00:56:25.895013 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.894986    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca" pod "0bd748a7-8da3-44d6-b26c-d3f3eb034c99" (UID: "0bd748a7-8da3-44d6-b26c-d3f3eb034c99"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:56:25.895187 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.895168    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:25.897283 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.897246    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt" pod "0bd748a7-8da3-44d6-b26c-d3f3eb034c99" (UID: "0bd748a7-8da3-44d6-b26c-d3f3eb034c99"). InnerVolumeSpecName "kube-api-access-hnqgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:56:25.897283 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.897255    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca" pod "0bd748a7-8da3-44d6-b26c-d3f3eb034c99" (UID: "0bd748a7-8da3-44d6-b26c-d3f3eb034c99"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:56:25.996339 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.996305    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnqgt\" (UniqueName: \"kubernetes.io/projected/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-kube-api-access-hnqgt\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:25.996339 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:25.996335    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0bd748a7-8da3-44d6-b26c-d3f3eb034c99-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:26.668816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:26.668782    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc" event={"ID":"0bd748a7-8da3-44d6-b26c-d3f3eb034c99","Type":"ContainerDied","Data":"b5bf6ad98fb13eb0a790a4d3efcdf27a1d9dfb34599d7221029d8569e5a63b91"}
Oct 30 00:56:26.669008 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:26.668828    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5bf6ad98fb13eb0a790a4d3efcdf27a1d9dfb34599d7221029d8569e5a63b91"
Oct 30 00:56:26.669008 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:26.668794    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"
Oct 30 00:56:26.915115 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:26.915083    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:27.013628 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:56:27.013587    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:56:28.701425 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:28.701391    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:29.572508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:29.572471    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:30.500898 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:30.500867    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:32.199576 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:32.199531    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:32.391402 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:32.391367    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:34.010398 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:34.010359    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:35.722812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:35.722771    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:37.511039 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:37.511000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:39.305275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:39.305234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:41.112119 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.112077    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:41.337518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.337482    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"]
Oct 30 00:56:41.338103 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.338083    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0bd748a7-8da3-44d6-b26c-d3f3eb034c99" containerName="revoke-keys"
Oct 30 00:56:41.338216 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.338104    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bd748a7-8da3-44d6-b26c-d3f3eb034c99" containerName="revoke-keys"
Oct 30 00:56:41.338271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.338243    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bd748a7-8da3-44d6-b26c-d3f3eb034c99" containerName="revoke-keys"
Oct 30 00:56:41.343193 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.343172    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.360371 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.360350    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"]
Oct 30 00:56:41.370406 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.370333    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:56:41.370406 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.370378    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:56:41.477157 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.477118    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.477346 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.477250    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.477346 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.477317    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7kl5\" (UniqueName: \"kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.578761 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.578724    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.578985 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.578795    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7kl5\" (UniqueName: \"kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.578985 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.578836    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.579603 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.579503    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.581468 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.581441    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.603606 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.603582    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7kl5\" (UniqueName: \"kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5\") pod \"maas-api-revoke-sub-dd465542-pvhz4\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.659593 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.659519    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:41.961692 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:41.961665    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"]
Oct 30 00:56:41.963893 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:56:41.963838    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22df6c37_ebc8_4e39_92a3_afb004f358f8.slice/crio-7c5bf658dc3ef061c65d5c8867ce03e9af4da216545a04e6b8a926c15b267e39 WatchSource:0}: Error finding container 7c5bf658dc3ef061c65d5c8867ce03e9af4da216545a04e6b8a926c15b267e39: Status 404 returned error can't find the container with id 7c5bf658dc3ef061c65d5c8867ce03e9af4da216545a04e6b8a926c15b267e39
Oct 30 00:56:42.774070 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:42.774036    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4" event={"ID":"22df6c37-ebc8-4e39-92a3-afb004f358f8","Type":"ContainerStarted","Data":"6d7aae0b57c7691a0ad855e49812ce39d5756e1134e5ff9d7632f8c6e9beb147"}
Oct 30 00:56:42.774458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:42.774084    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4" event={"ID":"22df6c37-ebc8-4e39-92a3-afb004f358f8","Type":"ContainerStarted","Data":"7c5bf658dc3ef061c65d5c8867ce03e9af4da216545a04e6b8a926c15b267e39"}
Oct 30 00:56:42.798170 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:42.798136    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:42.817591 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:42.817543    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4" podStartSLOduration=1.817531362 podStartE2EDuration="1.817531362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:56:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:56:42.81652778 +0000 UTC m=+910.845838010" watchObservedRunningTime="2026-10-30 00:56:42.817531362 +0000 UTC m=+910.846841629"
Oct 30 00:56:43.782139 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:43.782101    4577 generic.go:356] "Generic (PLEG): container finished" podID="22df6c37-ebc8-4e39-92a3-afb004f358f8" containerID="6d7aae0b57c7691a0ad855e49812ce39d5756e1134e5ff9d7632f8c6e9beb147" exitCode=0
Oct 30 00:56:43.782527 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:43.782165    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4" event={"ID":"22df6c37-ebc8-4e39-92a3-afb004f358f8","Type":"ContainerDied","Data":"6d7aae0b57c7691a0ad855e49812ce39d5756e1134e5ff9d7632f8c6e9beb147"}
Oct 30 00:56:44.500675 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:44.500592    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:44.933448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:44.933426    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:45.012800 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.012769    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca\") pod \"22df6c37-ebc8-4e39-92a3-afb004f358f8\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") "
Oct 30 00:56:45.013014 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.012887    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca\") pod \"22df6c37-ebc8-4e39-92a3-afb004f358f8\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") "
Oct 30 00:56:45.013014 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.012925    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5\" (UniqueName: \"kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5\") pod \"22df6c37-ebc8-4e39-92a3-afb004f358f8\" (UID: \"22df6c37-ebc8-4e39-92a3-afb004f358f8\") "
Oct 30 00:56:45.013241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.013204    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca" pod "22df6c37-ebc8-4e39-92a3-afb004f358f8" (UID: "22df6c37-ebc8-4e39-92a3-afb004f358f8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:56:45.015158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.015131    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5" pod "22df6c37-ebc8-4e39-92a3-afb004f358f8" (UID: "22df6c37-ebc8-4e39-92a3-afb004f358f8"). InnerVolumeSpecName "kube-api-access-p7kl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:56:45.015275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.015188    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca" pod "22df6c37-ebc8-4e39-92a3-afb004f358f8" (UID: "22df6c37-ebc8-4e39-92a3-afb004f358f8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:56:45.114904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.114806    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:45.114904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.114831    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7kl5\" (UniqueName: \"kubernetes.io/projected/22df6c37-ebc8-4e39-92a3-afb004f358f8-kube-api-access-p7kl5\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:45.114904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.114840    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22df6c37-ebc8-4e39-92a3-afb004f358f8-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:45.795671 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.795639    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"
Oct 30 00:56:45.795895 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.795673    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4" event={"ID":"22df6c37-ebc8-4e39-92a3-afb004f358f8","Type":"ContainerDied","Data":"7c5bf658dc3ef061c65d5c8867ce03e9af4da216545a04e6b8a926c15b267e39"}
Oct 30 00:56:45.795895 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:45.795722    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c5bf658dc3ef061c65d5c8867ce03e9af4da216545a04e6b8a926c15b267e39"
Oct 30 00:56:46.398387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:46.398357    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:48.221243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:48.221216    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:48.334373 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:48.334337    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:49.019289 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:49.019243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:50.003208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:50.003163    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:51.394651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.394620    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:51.824693 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.824664    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"]
Oct 30 00:56:51.825152 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.825140    4577 cpu_manager.go:414] "containerMap: removing container" podUID="22df6c37-ebc8-4e39-92a3-afb004f358f8" containerName="revoke-keys"
Oct 30 00:56:51.825195 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.825154    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22df6c37-ebc8-4e39-92a3-afb004f358f8" containerName="revoke-keys"
Oct 30 00:56:51.825258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.825249    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="22df6c37-ebc8-4e39-92a3-afb004f358f8" containerName="revoke-keys"
Oct 30 00:56:51.829658 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.829636    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:51.850453 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.850422    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"]
Oct 30 00:56:51.860467 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.860444    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:56:51.860735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.860714    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:56:51.913241 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.913213    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:51.913419 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.913378    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:51.913485 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.913467    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffbx\" (UniqueName: \"kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:51.933719 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:51.933695    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:52.014866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.014815    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffbx\" (UniqueName: \"kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.015070 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.014897    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.015070 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.015002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.015540 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.015519    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.017549 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.017527    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.039798 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.039770    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffbx\" (UniqueName: \"kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx\") pod \"maas-api-revoke-sub-e2d93b59-tzzks\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.145030 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.144956    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:52.451616 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.451573    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"]
Oct 30 00:56:52.453062 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:56:52.453041    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2299a7b9_1527_4349_927b_0df2e359c37a.slice/crio-ae882a6d9fb6d10957cf85ee563ef9d387a592f666ee33acc912d122e06f7e2f WatchSource:0}: Error finding container ae882a6d9fb6d10957cf85ee563ef9d387a592f666ee33acc912d122e06f7e2f: Status 404 returned error can't find the container with id ae882a6d9fb6d10957cf85ee563ef9d387a592f666ee33acc912d122e06f7e2f
Oct 30 00:56:52.842011 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.841971    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks" event={"ID":"2299a7b9-1527-4349-927b-0df2e359c37a","Type":"ContainerStarted","Data":"713a7c70f0816b4905f2111c51b4f9b656a194ed32ebad68c43d3e97ab579b30"}
Oct 30 00:56:52.842201 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.842025    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks" event={"ID":"2299a7b9-1527-4349-927b-0df2e359c37a","Type":"ContainerStarted","Data":"ae882a6d9fb6d10957cf85ee563ef9d387a592f666ee33acc912d122e06f7e2f"}
Oct 30 00:56:52.886312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:52.886267    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks" podStartSLOduration=1.886253447 podStartE2EDuration="1.886253447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:56:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:56:52.885071039 +0000 UTC m=+920.914381270" watchObservedRunningTime="2026-10-30 00:56:52.886253447 +0000 UTC m=+920.915563677"
Oct 30 00:56:53.757254 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:53.757198    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:53.852546 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:53.852507    4577 generic.go:356] "Generic (PLEG): container finished" podID="2299a7b9-1527-4349-927b-0df2e359c37a" containerID="713a7c70f0816b4905f2111c51b4f9b656a194ed32ebad68c43d3e97ab579b30" exitCode=0
Oct 30 00:56:53.852759 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:53.852593    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks" event={"ID":"2299a7b9-1527-4349-927b-0df2e359c37a","Type":"ContainerDied","Data":"713a7c70f0816b4905f2111c51b4f9b656a194ed32ebad68c43d3e97ab579b30"}
Oct 30 00:56:55.020927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.020899    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:55.147812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.147721    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx\" (UniqueName: \"kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx\") pod \"2299a7b9-1527-4349-927b-0df2e359c37a\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") "
Oct 30 00:56:55.147812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.147786    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca\") pod \"2299a7b9-1527-4349-927b-0df2e359c37a\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") "
Oct 30 00:56:55.148068 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.147878    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca\") pod \"2299a7b9-1527-4349-927b-0df2e359c37a\" (UID: \"2299a7b9-1527-4349-927b-0df2e359c37a\") "
Oct 30 00:56:55.148190 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.148168    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca" pod "2299a7b9-1527-4349-927b-0df2e359c37a" (UID: "2299a7b9-1527-4349-927b-0df2e359c37a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:56:55.148376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.148360    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:55.150108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.150076    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca" pod "2299a7b9-1527-4349-927b-0df2e359c37a" (UID: "2299a7b9-1527-4349-927b-0df2e359c37a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:56:55.150402 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.150378    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx" pod "2299a7b9-1527-4349-927b-0df2e359c37a" (UID: "2299a7b9-1527-4349-927b-0df2e359c37a"). InnerVolumeSpecName "kube-api-access-sffbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:56:55.249906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.249839    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sffbx\" (UniqueName: \"kubernetes.io/projected/2299a7b9-1527-4349-927b-0df2e359c37a-kube-api-access-sffbx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:55.249906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.249907    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2299a7b9-1527-4349-927b-0df2e359c37a-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:56:55.531962 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.531932    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:55.867812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.867721    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks" event={"ID":"2299a7b9-1527-4349-927b-0df2e359c37a","Type":"ContainerDied","Data":"ae882a6d9fb6d10957cf85ee563ef9d387a592f666ee33acc912d122e06f7e2f"}
Oct 30 00:56:55.867812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.867768    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae882a6d9fb6d10957cf85ee563ef9d387a592f666ee33acc912d122e06f7e2f"
Oct 30 00:56:55.867812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:55.867738    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"
Oct 30 00:56:57.016871 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:56:57.016311    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:56:57.607701 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:57.607668    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:56:59.421428 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:56:59.421392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:01.113866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:01.113796    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:02.834295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:02.834257    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:04.552024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:04.551983    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:06.329454 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:06.329419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:08.009490 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.009456    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"]
Oct 30 00:57:08.010304 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.010277    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2299a7b9-1527-4349-927b-0df2e359c37a" containerName="revoke-keys"
Oct 30 00:57:08.010421 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.010307    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2299a7b9-1527-4349-927b-0df2e359c37a" containerName="revoke-keys"
Oct 30 00:57:08.010470 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.010423    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2299a7b9-1527-4349-927b-0df2e359c37a" containerName="revoke-keys"
Oct 30 00:57:08.014686 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.014666    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.023544 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.023151    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"]
Oct 30 00:57:08.033273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.033253    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"kube-root-ca.crt\""
Oct 30 00:57:08.048106 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.048078    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"e2e-conflict-b-988896f8-kserve-self-signed-certs\""
Oct 30 00:57:08.048249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.048173    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"openshift-service-ca.crt\""
Oct 30 00:57:08.048249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.048242    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"default-dockercfg-nf2tx\""
Oct 30 00:57:08.064454 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.064374    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:08.149418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.149385    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.149418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.149422    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.149640 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.149444    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnjdd\" (UniqueName: \"kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.149640 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.149461    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.149640 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.149563    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.149640 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.149604    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.193128 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.192298    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:08.251124 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251089    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251141    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251174    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnjdd\" (UniqueName: \"kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251197    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251227    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251312 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251264    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251685 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251644    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251685 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251663    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.251884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.251708    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.253980 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.253960    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.254161 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.254142    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.276093 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.276039    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnjdd\" (UniqueName: \"kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd\") pod \"e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") " pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.331419 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.331394    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:08.686007 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.685980    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"]
Oct 30 00:57:08.960897 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.960794    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8" event={"ID":"7088281a-1be9-43eb-bbec-dfad2fcb10a2","Type":"ContainerStarted","Data":"e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838"}
Oct 30 00:57:08.960897 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:08.960856    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8" event={"ID":"7088281a-1be9-43eb-bbec-dfad2fcb10a2","Type":"ContainerStarted","Data":"b7ad4f2173134390983adf7335d1c684f6d902ed36da974ec544a78da0ba2990"}
Oct 30 00:57:09.006602 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:09.006553    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8" podStartSLOduration=2.006537085 podStartE2EDuration="2.006537085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:57:09.005611322 +0000 UTC m=+937.034921554" watchObservedRunningTime="2026-10-30 00:57:09.006537085 +0000 UTC m=+937.035847315"
Oct 30 00:57:09.331748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:09.331719    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:10.012164 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:10.012132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:11.697921 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:11.697885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:12.572272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.572176    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"]
Oct 30 00:57:12.576161 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.576141    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.605313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.605276    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"]
Oct 30 00:57:12.705043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705010    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vvff\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705046    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705066    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705084    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705220    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705359    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705385    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705417    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.705642 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.705466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806450    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vvff\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806495    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806524    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806548    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806578    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806690    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.806750 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806714    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806750    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.806795    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.807005    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807209 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.807063    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.807205    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807319 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.807266    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.807483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.807463    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.809183 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.809160    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.809307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.809272    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.831676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.831604    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.831927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.831905    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vvff\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:12.891531 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:12.891499    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:13.210890 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:13.210864    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"]
Oct 30 00:57:13.211468 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:57:13.211443    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6b4213_198c_4ba3_b56a_09fe015ae4d8.slice/crio-85ea8d4f04d434ef1ba399301d499ffdfa9a7439fe33cb9421fb048ee20a140c WatchSource:0}: Error finding container 85ea8d4f04d434ef1ba399301d499ffdfa9a7439fe33cb9421fb048ee20a140c: Status 404 returned error can't find the container with id 85ea8d4f04d434ef1ba399301d499ffdfa9a7439fe33cb9421fb048ee20a140c
Oct 30 00:57:13.213829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:13.213800    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:57:13.213936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:13.213882    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:57:13.213936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:13.213916    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:57:13.417181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:13.417149    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:13.532300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:13.532264    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:14.002746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:14.001590    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" event={"ID":"cc6b4213-198c-4ba3-b56a-09fe015ae4d8","Type":"ContainerStarted","Data":"e0bfdcb11e960cd7169f39efe33aee0957e3cc8daf7da5a18c0db5333a5c4f5c"}
Oct 30 00:57:14.002746 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:14.001669    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" event={"ID":"cc6b4213-198c-4ba3-b56a-09fe015ae4d8","Type":"ContainerStarted","Data":"85ea8d4f04d434ef1ba399301d499ffdfa9a7439fe33cb9421fb048ee20a140c"}
Oct 30 00:57:14.892236 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:14.892199    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:14.895191 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:14.894993    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:57:14.895191 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:14.895044    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:57:15.327758 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:15.327725    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:15.892580 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:15.892537    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:57:15.892803 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:15.892596    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:57:16.892725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:16.892268    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:57:16.892725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:16.892332    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:57:17.158617 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:17.158463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:17.892305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:17.892254    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:57:17.892790 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:17.892760    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:57:18.197207 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.197078    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podStartSLOduration=6.197056516 podStartE2EDuration="6.197056516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:57:14.084197887 +0000 UTC m=+942.113508155" watchObservedRunningTime="2026-10-30 00:57:18.197056516 +0000 UTC m=+946.226366748"
Oct 30 00:57:18.205278 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.205232    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"]
Oct 30 00:57:18.241142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.241096    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"]
Oct 30 00:57:18.241376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.241259    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.321805 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.321764    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.321881    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.321932    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.321975    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322153 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.322091    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322153 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.322140    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322228 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.322172    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.322248    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.322271 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.322263    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4vlp\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.332527 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.332504    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:18.351391 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.351124    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:18.423456 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423421    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.423660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423469    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.423660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423486    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4vlp\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.423660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423550    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.423660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423572    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.423660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423601    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.423660 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423641    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423706    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423769    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.423948    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424207 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.424187    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424297 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.424279    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424370 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.424328    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.424425 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.424388    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.426426 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.426396    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.426737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.426711    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.472915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.469038    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4vlp\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.472915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.469277    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.602937 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.602901    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:18.892275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.892242    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:57:18.892436 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.892309    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:57:18.965491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.965449    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:18.971115 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:57:18.966137    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aaea6a7_d2fa_4d40_9a9d_cb46a3044cb2.slice/crio-3b0fc8b43d9f6bc50bb8019b774f167b967f60d270ec976a4399bc39f51f464d WatchSource:0}: Error finding container 3b0fc8b43d9f6bc50bb8019b774f167b967f60d270ec976a4399bc39f51f464d: Status 404 returned error can't find the container with id 3b0fc8b43d9f6bc50bb8019b774f167b967f60d270ec976a4399bc39f51f464d
Oct 30 00:57:18.972457 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.972426    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"]
Oct 30 00:57:18.977149 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.977113    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:57:18.977235 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.977199    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:57:18.977279 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:18.977244    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:57:19.039237 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:19.039202    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" event={"ID":"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2","Type":"ContainerStarted","Data":"3b0fc8b43d9f6bc50bb8019b774f167b967f60d270ec976a4399bc39f51f464d"}
Oct 30 00:57:19.082201 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:19.080775    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:19.477455 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:19.477422    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:19.892634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:19.892595    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:57:19.892814 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:19.892687    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:57:20.061100 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.061067    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" event={"ID":"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2","Type":"ContainerStarted","Data":"376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7"}
Oct 30 00:57:20.167921 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.166921    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podStartSLOduration=2.16690148 podStartE2EDuration="2.16690148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:57:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:57:20.124550394 +0000 UTC m=+948.153860626" watchObservedRunningTime="2026-10-30 00:57:20.16690148 +0000 UTC m=+948.196211712"
Oct 30 00:57:20.604112 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.604077    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:20.605776 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.605748    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:57:20.605936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.605799    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:57:20.802289 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.802255    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:20.923197 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.923109    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:20.923572 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.923537    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:20.924569 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:20.924551    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 00:57:21.604086 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:21.604052    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:57:21.604597 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:21.604112    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:57:22.603418 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:22.603383    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:57:22.603608 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:22.603435    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:57:22.605032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:22.605005    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:23.604044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:23.604012    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:57:23.604235 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:23.604063    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:57:24.394084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:24.394049    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:24.603752 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:24.603717    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:57:24.603933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:24.603777    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:57:25.995074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:25.995030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:25.999929 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:25.999643    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:26.000970 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:26.000939    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:26.002581 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:26.002542    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 00:57:26.122238 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:26.122183    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:27.023789 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:57:27.023747    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:57:27.805654 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:27.805620    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:29.600486 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:29.600451    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:29.865643 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:29.865543    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:31.412243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:31.412211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:33.109604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:33.109549    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:34.801262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:34.801231    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:35.148488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:35.148406    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:36.023986 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:36.023949    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:36.610334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:36.610288    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:38.421683 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:38.421630    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:40.200774 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:40.200725    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:41.976568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:41.976528    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:43.577295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:43.577265    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"]
Oct 30 00:57:43.617173 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:43.617138    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:43.736062 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:43.736022    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:45.297436 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:45.297405    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:46.968679 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:46.968637    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:47.730322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:47.730293    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:48.810099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:48.810072    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:50.513459 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:50.513417    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:52.268263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:52.268229    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:54.010793 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:54.010759    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:55.762107 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:55.762006    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:57.015580 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:57:57.015544    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:57:57.556260 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:57.556220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:58.598574 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.598488    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8" podUID="7088281a-1be9-43eb-bbec-dfad2fcb10a2" containerName="main" containerID="cri-o://e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838" gracePeriod=45
Oct 30 00:57:58.872808 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.872785    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:58.978837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.978804    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm\") pod \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") "
Oct 30 00:57:58.978837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.978872    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd\" (UniqueName: \"kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd\") pod \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") "
Oct 30 00:57:58.979143 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979084    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs\" (UniqueName: \"kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs\") pod \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") "
Oct 30 00:57:58.979143 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979116    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home\") pod \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") "
Oct 30 00:57:58.979251 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979181    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache\") pod \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") "
Oct 30 00:57:58.979251 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979237    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir\") pod \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\" (UID: \"7088281a-1be9-43eb-bbec-dfad2fcb10a2\") "
Oct 30 00:57:58.979521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979463    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home" pod "7088281a-1be9-43eb-bbec-dfad2fcb10a2" (UID: "7088281a-1be9-43eb-bbec-dfad2fcb10a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:57:58.979824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979536    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir" pod "7088281a-1be9-43eb-bbec-dfad2fcb10a2" (UID: "7088281a-1be9-43eb-bbec-dfad2fcb10a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:57:58.979824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979643    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache" pod "7088281a-1be9-43eb-bbec-dfad2fcb10a2" (UID: "7088281a-1be9-43eb-bbec-dfad2fcb10a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:57:58.979824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979727    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:57:58.979824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979748    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:57:58.979824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.979761    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:57:58.981495 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.981472    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm" pod "7088281a-1be9-43eb-bbec-dfad2fcb10a2" (UID: "7088281a-1be9-43eb-bbec-dfad2fcb10a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:57:58.981611 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.981551    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd" pod "7088281a-1be9-43eb-bbec-dfad2fcb10a2" (UID: "7088281a-1be9-43eb-bbec-dfad2fcb10a2"). InnerVolumeSpecName "kube-api-access-lnjdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:57:58.981657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:58.981619    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs" pod "7088281a-1be9-43eb-bbec-dfad2fcb10a2" (UID: "7088281a-1be9-43eb-bbec-dfad2fcb10a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:57:59.080607 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.080572    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7088281a-1be9-43eb-bbec-dfad2fcb10a2-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:57:59.080607 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.080602    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7088281a-1be9-43eb-bbec-dfad2fcb10a2-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:57:59.080607 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.080612    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnjdd\" (UniqueName: \"kubernetes.io/projected/7088281a-1be9-43eb-bbec-dfad2fcb10a2-kube-api-access-lnjdd\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:57:59.255033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.254992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:57:59.517355 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.517269    4577 generic.go:356] "Generic (PLEG): container finished" podID="7088281a-1be9-43eb-bbec-dfad2fcb10a2" containerID="e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838" exitCode=0
Oct 30 00:57:59.517355 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.517318    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"
Oct 30 00:57:59.517547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.517369    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8" event={"ID":"7088281a-1be9-43eb-bbec-dfad2fcb10a2","Type":"ContainerDied","Data":"e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838"}
Oct 30 00:57:59.517547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.517418    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8" event={"ID":"7088281a-1be9-43eb-bbec-dfad2fcb10a2","Type":"ContainerDied","Data":"b7ad4f2173134390983adf7335d1c684f6d902ed36da974ec544a78da0ba2990"}
Oct 30 00:57:59.517547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.517437    4577 scope.go:122] "RemoveContainer" containerID="e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838"
Oct 30 00:57:59.528401 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.528379    4577 scope.go:122] "RemoveContainer" containerID="e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838"
Oct 30 00:57:59.528682 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:57:59.528663    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838\": container with ID starting with e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838 not found: ID does not exist" containerID="e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838"
Oct 30 00:57:59.528748 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.528692    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838"} err="failed to get container status \"e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838\": rpc error: code = NotFound desc = could not find container \"e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838\": container with ID starting with e27166cc6523ab8db4a9d92c3e5be7c9def04c1701b4ca38cbcbf08e6e9f3838 not found: ID does not exist"
Oct 30 00:57:59.575367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.575334    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"]
Oct 30 00:57:59.589402 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:57:59.589370    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/e2e-conflict-b-988896f8-kserve-7bd7f97677-n54p8"]
Oct 30 00:58:00.112230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.112189    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7088281a-1be9-43eb-bbec-dfad2fcb10a2" path="/var/lib/kubelet/pods/7088281a-1be9-43eb-bbec-dfad2fcb10a2/volumes"
Oct 30 00:58:00.629927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.629891    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"]
Oct 30 00:58:00.630427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.630411    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7088281a-1be9-43eb-bbec-dfad2fcb10a2" containerName="main"
Oct 30 00:58:00.630427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.630427    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7088281a-1be9-43eb-bbec-dfad2fcb10a2" containerName="main"
Oct 30 00:58:00.630577 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.630535    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7088281a-1be9-43eb-bbec-dfad2fcb10a2" containerName="main"
Oct 30 00:58:00.635618 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.635594    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.679835 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.679805    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-d5b2b8\"/\"openshift-service-ca.crt\""
Oct 30 00:58:00.680097 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.679813    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-d5b2b8\"/\"distinct-2-w4-d5b2b8-kserve-self-signed-certs\""
Oct 30 00:58:00.680181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.679827    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-d5b2b8\"/\"kube-root-ca.crt\""
Oct 30 00:58:00.680181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.679883    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-d5b2b8\"/\"default-dockercfg-f9x4g\""
Oct 30 00:58:00.684695 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.684672    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"]
Oct 30 00:58:00.697170 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.697144    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.697329 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.697180    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnd79\" (UniqueName: \"kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.697329 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.697270    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.697465 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.697398    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.697532 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.697458    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.697582 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.697556    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799145    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799200    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799235    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799267    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799336    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799367    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnd79\" (UniqueName: \"kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799664 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799603    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799664 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799612    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.799754 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.799668    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.801752 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.801726    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.802079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.802060    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.803725 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.803704    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:00.824860 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.824815    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnd79\" (UniqueName: \"kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79\") pod \"distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") " pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.950146 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.950054    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:00.978266 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:00.978230    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:01.261445 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:01.261412    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e225ef3_81a7_4fce_a53a_0d1edc73487f.slice/crio-bd1027d0bf1cc9df1539634b595bd1e1d908c97c9b30c1f8d200f69bf65c8975 WatchSource:0}: Error finding container bd1027d0bf1cc9df1539634b595bd1e1d908c97c9b30c1f8d200f69bf65c8975: Status 404 returned error can't find the container with id bd1027d0bf1cc9df1539634b595bd1e1d908c97c9b30c1f8d200f69bf65c8975
Oct 30 00:58:01.262311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:01.262282    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"]
Oct 30 00:58:01.534116 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:01.534029    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" event={"ID":"7e225ef3-81a7-4fce-a53a-0d1edc73487f","Type":"ContainerStarted","Data":"564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239"}
Oct 30 00:58:01.534116 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:01.534070    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" event={"ID":"7e225ef3-81a7-4fce-a53a-0d1edc73487f","Type":"ContainerStarted","Data":"bd1027d0bf1cc9df1539634b595bd1e1d908c97c9b30c1f8d200f69bf65c8975"}
Oct 30 00:58:01.580182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:01.580133    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podStartSLOduration=1.580118243 podStartE2EDuration="1.580118243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:01.578337517 +0000 UTC m=+989.607647747" watchObservedRunningTime="2026-10-30 00:58:01.580118243 +0000 UTC m=+989.609428473"
Oct 30 00:58:01.951062 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:01.950938    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:02.740534 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:02.740500    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:04.511052 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.511019    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:04.708554 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.708523    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"]
Oct 30 00:58:04.712518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.712501    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.731565 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.731534    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"]
Oct 30 00:58:04.740377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.740343    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:58:04.740517 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.740456    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:58:04.851009 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.850908    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.851009 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.850986    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.851254 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.851086    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jss9b\" (UniqueName: \"kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.952558 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.952524    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.952558 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.952564    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jss9b\" (UniqueName: \"kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.952811 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.952658    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.953300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.953277    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.955116 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.955092    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:04.978939 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:04.978915    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jss9b\" (UniqueName: \"kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b\") pod \"maas-api-revoke-sub-9b3887ff-mwf9f\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:05.026896 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:05.026836    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:05.332742 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:05.332710    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"]
Oct 30 00:58:05.333041 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:05.333018    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de41d5d_743f_4b07_8b08_b80fc05fdf5e.slice/crio-7885502233d79623e2c24b65bcc591269ff10cf892cdec65f36f0af403604c54 WatchSource:0}: Error finding container 7885502233d79623e2c24b65bcc591269ff10cf892cdec65f36f0af403604c54: Status 404 returned error can't find the container with id 7885502233d79623e2c24b65bcc591269ff10cf892cdec65f36f0af403604c54
Oct 30 00:58:05.607300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:05.607214    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f" event={"ID":"4de41d5d-743f-4b07-8b08-b80fc05fdf5e","Type":"ContainerStarted","Data":"7885502233d79623e2c24b65bcc591269ff10cf892cdec65f36f0af403604c54"}
Oct 30 00:58:06.425071 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.425026    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:06.616087 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.616054    4577 generic.go:356] "Generic (PLEG): container finished" podID="4de41d5d-743f-4b07-8b08-b80fc05fdf5e" containerID="f6aaeda0bb85095b10f5e25ffd6d2ee3386d5810cf6c413a44144e92a2a57b47" exitCode=0
Oct 30 00:58:06.616547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.616139    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f" event={"ID":"4de41d5d-743f-4b07-8b08-b80fc05fdf5e","Type":"ContainerDied","Data":"f6aaeda0bb85095b10f5e25ffd6d2ee3386d5810cf6c413a44144e92a2a57b47"}
Oct 30 00:58:06.677699 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.677654    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"]
Oct 30 00:58:06.681674 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.681649    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.699621 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.699585    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"]
Oct 30 00:58:06.778376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778318    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778580 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778484    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778688 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778669    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778710    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778769 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778733    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778870    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778919    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2rt\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778940    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.778974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.778963    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880234    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880309    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880347    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880392    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880444    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2rt\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880476    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880504    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880551    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880617    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880698    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881019 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.880988    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881563 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.881041    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881563 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.881261    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.881563 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.881332    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.883833 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.883808    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.884341 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.884322    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.908910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.908881    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.909076 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.909000    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2rt\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:06.914415 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:06.914388    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:07.001516 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.001475    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:07.462834 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.462802    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"]
Oct 30 00:58:07.465767 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:07.465733    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f89417c_ccc2_40ca_a13f_6014e47ea558.slice/crio-6d008e22b5c39c54ed31bfe4e5935c70f2aa25aba23096bc8fb5475d4aa3b029 WatchSource:0}: Error finding container 6d008e22b5c39c54ed31bfe4e5935c70f2aa25aba23096bc8fb5475d4aa3b029: Status 404 returned error can't find the container with id 6d008e22b5c39c54ed31bfe4e5935c70f2aa25aba23096bc8fb5475d4aa3b029
Oct 30 00:58:07.469337 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.469298    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:07.469466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.469403    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:07.469466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.469449    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:07.670870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.670091    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" event={"ID":"7f89417c-ccc2-40ca-a13f-6014e47ea558","Type":"ContainerStarted","Data":"8df9b9e02eb4ac3af9a0bdfc6c681b61b43879b5a5e032c75aa27884dfb67c13"}
Oct 30 00:58:07.670870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.670146    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" event={"ID":"7f89417c-ccc2-40ca-a13f-6014e47ea558","Type":"ContainerStarted","Data":"6d008e22b5c39c54ed31bfe4e5935c70f2aa25aba23096bc8fb5475d4aa3b029"}
Oct 30 00:58:07.776872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:07.775809    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podStartSLOduration=1.775789283 podStartE2EDuration="1.775789283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:07.766011421 +0000 UTC m=+995.795321654" watchObservedRunningTime="2026-10-30 00:58:07.775789283 +0000 UTC m=+995.805099515"
Oct 30 00:58:08.025935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.024281    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:08.025935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.024424    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:08.025935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.024468    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:08.086921 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.084123    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:08.253910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.235257    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca\") pod \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") "
Oct 30 00:58:08.253910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.235406    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca\") pod \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") "
Oct 30 00:58:08.253910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.235524    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b\" (UniqueName: \"kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b\") pod \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\" (UID: \"4de41d5d-743f-4b07-8b08-b80fc05fdf5e\") "
Oct 30 00:58:08.276870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.264054    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca" pod "4de41d5d-743f-4b07-8b08-b80fc05fdf5e" (UID: "4de41d5d-743f-4b07-8b08-b80fc05fdf5e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:58:08.304072 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.277422    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca" pod "4de41d5d-743f-4b07-8b08-b80fc05fdf5e" (UID: "4de41d5d-743f-4b07-8b08-b80fc05fdf5e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:58:08.305094 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.305068    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:08.305321 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.305301    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:08.309132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.309083    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b" pod "4de41d5d-743f-4b07-8b08-b80fc05fdf5e" (UID: "4de41d5d-743f-4b07-8b08-b80fc05fdf5e"). InnerVolumeSpecName "kube-api-access-jss9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:58:08.406872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.406579    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jss9b\" (UniqueName: \"kubernetes.io/projected/4de41d5d-743f-4b07-8b08-b80fc05fdf5e-kube-api-access-jss9b\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:08.494869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.488590    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:08.751871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.735537    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"
Oct 30 00:58:08.755869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.752978    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f" event={"ID":"4de41d5d-743f-4b07-8b08-b80fc05fdf5e","Type":"ContainerDied","Data":"7885502233d79623e2c24b65bcc591269ff10cf892cdec65f36f0af403604c54"}
Oct 30 00:58:08.755869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:08.753062    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7885502233d79623e2c24b65bcc591269ff10cf892cdec65f36f0af403604c54"
Oct 30 00:58:09.002933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:09.002283    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:09.002933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:09.002364    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:10.002719 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:10.002686    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:10.003159 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:10.002735    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:10.277763 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:10.277647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:10.958867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:10.958810    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:10.998935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:10.993433    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:11.002420 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:11.002382    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:11.002557 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:11.002437    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:11.787756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:11.787724    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:12.002395 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:12.002363    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:12.002585 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:12.002418    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:12.111092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:12.111005    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:13.002452 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:13.002415    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:13.002976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:13.002479    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:13.905479 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:13.905452    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:14.006816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:14.006266    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:58:14.006816 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:14.006328    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:58:15.006099 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:15.006074    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:15.006525 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:15.006508    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:15.007301 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:15.007282    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 00:58:15.715585 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:15.715550    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:17.513448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:17.513412    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:19.212878 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.212831    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:19.591880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.588289    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:19.660261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.660217    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"]
Oct 30 00:58:19.660964 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.660942    4577 cpu_manager.go:414] "containerMap: removing container" podUID="4de41d5d-743f-4b07-8b08-b80fc05fdf5e" containerName="revoke-keys"
Oct 30 00:58:19.660964 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.660962    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4de41d5d-743f-4b07-8b08-b80fc05fdf5e" containerName="revoke-keys"
Oct 30 00:58:19.661150 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.661059    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="4de41d5d-743f-4b07-8b08-b80fc05fdf5e" containerName="revoke-keys"
Oct 30 00:58:19.664756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.664738    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.684500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.684471    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"]
Oct 30 00:58:19.692780 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.692757    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:58:19.692948 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.692886    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:58:19.719704 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.719673    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.719925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.719709    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k44cp\" (UniqueName: \"kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.719925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.719794    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.820694 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.820660    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.820694 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.820705    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k44cp\" (UniqueName: \"kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.821022 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.820762    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.821708 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.821675    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.823377 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.823353    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.846758 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.846688    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k44cp\" (UniqueName: \"kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp\") pod \"maas-api-revoke-sub-5c283c5e-9njvv\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:19.978622 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:19.978579    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:20.284571 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.284533    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"]
Oct 30 00:58:20.285069 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:20.285008    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1c6877_8cd5_4114_adfa_b9f69612aab9.slice/crio-21b16e4bc4c5ff45158f78a05c38dee1a84ad52546277f0894bf06f0c6cbbe9b WatchSource:0}: Error finding container 21b16e4bc4c5ff45158f78a05c38dee1a84ad52546277f0894bf06f0c6cbbe9b: Status 404 returned error can't find the container with id 21b16e4bc4c5ff45158f78a05c38dee1a84ad52546277f0894bf06f0c6cbbe9b
Oct 30 00:58:20.846247 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.846209    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv" event={"ID":"fc1c6877-8cd5-4114-adfa-b9f69612aab9","Type":"ContainerStarted","Data":"aef49c3cbb57b6e3c244410d1948e88af67cac482cd53a3ba63d86996b5fa3ad"}
Oct 30 00:58:20.846429 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.846307    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv" event={"ID":"fc1c6877-8cd5-4114-adfa-b9f69612aab9","Type":"ContainerStarted","Data":"21b16e4bc4c5ff45158f78a05c38dee1a84ad52546277f0894bf06f0c6cbbe9b"}
Oct 30 00:58:20.888416 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.888380    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"]
Oct 30 00:58:20.895815 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.895747    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:20.896254 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.895907    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv" podStartSLOduration=1.895890379 podStartE2EDuration="1.895890379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:20.895208599 +0000 UTC m=+1008.924518822" watchObservedRunningTime="2026-10-30 00:58:20.895890379 +0000 UTC m=+1008.925200610"
Oct 30 00:58:20.910082 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.910049    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"]
Oct 30 00:58:20.912966 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.912939    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"openshift-service-ca.crt\""
Oct 30 00:58:20.913098 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.913046    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"kube-root-ca.crt\""
Oct 30 00:58:20.913098 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.913094    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"default-dockercfg-nf2tx\""
Oct 30 00:58:20.913172 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.913155    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-39b241\"/\"distinct-2-w3-39b241-kserve-self-signed-certs\""
Oct 30 00:58:20.936243 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:20.936206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:21.041120 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.041090    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn79d\" (UniqueName: \"kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.041120 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.041122    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.041376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.041148    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.041376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.041244    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.041376 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.041291    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.041530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.041399    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.047306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.047284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:21.142778 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.142743    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143023 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.142804    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143023 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.142989    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn79d\" (UniqueName: \"kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143023 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.143017    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143217 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.143052    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143217 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.143091    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143361 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.143333    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.143480    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.143698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.143674    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.150262 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.150238    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.151013 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.150993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.168788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.168734    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn79d\" (UniqueName: \"kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d\") pod \"distinct-2-w3-39b241-kserve-6f5584d658-5hn44\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") " pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.212828 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.212802    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:21.517488 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:21.517443    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44192915_065f_4bca_820e_e901202dfb53.slice/crio-f8c024087340d52e762b7029fdc85fac47be4df608ae58b4efea01bab3343971 WatchSource:0}: Error finding container f8c024087340d52e762b7029fdc85fac47be4df608ae58b4efea01bab3343971: Status 404 returned error can't find the container with id f8c024087340d52e762b7029fdc85fac47be4df608ae58b4efea01bab3343971
Oct 30 00:58:21.517488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.517467    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"]
Oct 30 00:58:21.858168 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.858085    4577 generic.go:356] "Generic (PLEG): container finished" podID="fc1c6877-8cd5-4114-adfa-b9f69612aab9" containerID="aef49c3cbb57b6e3c244410d1948e88af67cac482cd53a3ba63d86996b5fa3ad" exitCode=0
Oct 30 00:58:21.858336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.858168    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv" event={"ID":"fc1c6877-8cd5-4114-adfa-b9f69612aab9","Type":"ContainerDied","Data":"aef49c3cbb57b6e3c244410d1948e88af67cac482cd53a3ba63d86996b5fa3ad"}
Oct 30 00:58:21.860048 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.860018    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" event={"ID":"44192915-065f-4bca-820e-e901202dfb53","Type":"ContainerStarted","Data":"5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6"}
Oct 30 00:58:21.860163 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.860052    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" event={"ID":"44192915-065f-4bca-820e-e901202dfb53","Type":"ContainerStarted","Data":"f8c024087340d52e762b7029fdc85fac47be4df608ae58b4efea01bab3343971"}
Oct 30 00:58:21.953322 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:21.953279    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podStartSLOduration=1.9532655330000002 podStartE2EDuration="1.953265533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:21.953020133 +0000 UTC m=+1009.982330362" watchObservedRunningTime="2026-10-30 00:58:21.953265533 +0000 UTC m=+1009.982575764"
Oct 30 00:58:22.213475 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:22.213397    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:22.716296 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:22.716265    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:23.021665 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.021640    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:23.166756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.166718    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca\") pod \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") "
Oct 30 00:58:23.166974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.166904    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp\" (UniqueName: \"kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp\") pod \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") "
Oct 30 00:58:23.166974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.166962    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca\") pod \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\" (UID: \"fc1c6877-8cd5-4114-adfa-b9f69612aab9\") "
Oct 30 00:58:23.167092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.167071    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca" pod "fc1c6877-8cd5-4114-adfa-b9f69612aab9" (UID: "fc1c6877-8cd5-4114-adfa-b9f69612aab9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:58:23.167703 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.167677    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:23.169222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.169195    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca" pod "fc1c6877-8cd5-4114-adfa-b9f69612aab9" (UID: "fc1c6877-8cd5-4114-adfa-b9f69612aab9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:58:23.169358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.169271    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp" pod "fc1c6877-8cd5-4114-adfa-b9f69612aab9" (UID: "fc1c6877-8cd5-4114-adfa-b9f69612aab9"). InnerVolumeSpecName "kube-api-access-k44cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:58:23.269362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.269273    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fc1c6877-8cd5-4114-adfa-b9f69612aab9-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:23.269362 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.269298    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k44cp\" (UniqueName: \"kubernetes.io/projected/fc1c6877-8cd5-4114-adfa-b9f69612aab9-kube-api-access-k44cp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:23.876994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.876941    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"
Oct 30 00:58:23.877387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.876962    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv" event={"ID":"fc1c6877-8cd5-4114-adfa-b9f69612aab9","Type":"ContainerDied","Data":"21b16e4bc4c5ff45158f78a05c38dee1a84ad52546277f0894bf06f0c6cbbe9b"}
Oct 30 00:58:23.877387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:23.877106    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21b16e4bc4c5ff45158f78a05c38dee1a84ad52546277f0894bf06f0c6cbbe9b"
Oct 30 00:58:24.519494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:24.519454    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:26.821870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:26.820073    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:27.023073 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:58:27.023028    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:58:27.801201 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.801172    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"]
Oct 30 00:58:27.801698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.801682    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fc1c6877-8cd5-4114-adfa-b9f69612aab9" containerName="revoke-keys"
Oct 30 00:58:27.801772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.801700    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc1c6877-8cd5-4114-adfa-b9f69612aab9" containerName="revoke-keys"
Oct 30 00:58:27.801810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.801806    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc1c6877-8cd5-4114-adfa-b9f69612aab9" containerName="revoke-keys"
Oct 30 00:58:27.805549 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.805527    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.844166 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.844129    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"]
Oct 30 00:58:27.901272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901238    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901463 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901308    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901517 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901499    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901581 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901563    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901637 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901622    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901722 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901707    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901771 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901739    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901771 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901762    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:27.901902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:27.901884    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k44h2\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003211    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k44h2\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003477 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003301    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003477 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003357    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003477 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003434    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003481    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003531    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003651 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003638    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003666    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003694    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003958 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003799    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.003958 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.003898    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.004131 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.004095    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.004200 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.004141    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.004599 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.004576    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.005812 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.005793    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.006057 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.006039    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.028095 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.028071    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.028402 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.028383    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k44h2\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.120212 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.120133    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"]
Oct 30 00:58:28.123408 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.123384    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:28.125478 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.125461    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.142364 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.142337    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:58:28.142466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.142454    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:58:28.142507 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.142468    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"]
Oct 30 00:58:28.206891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.206646    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4974\" (UniqueName: \"kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.206891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.206705    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.206891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.206795    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.308076 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.308049    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.308250 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.308116    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4974\" (UniqueName: \"kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.308250 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.308159    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.308764 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.308744    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.310560 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.310542    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.333403 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.333383    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4974\" (UniqueName: \"kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974\") pod \"maas-api-revoke-sub-47047b78-bk59w\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.452620 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.452595    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"]
Oct 30 00:58:28.452798 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:28.452778    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0856659_4315_48f2_be0f_def689ca8af3.slice/crio-8a8ba8618bb4e7505ef39257fc27bd5b336831cb6c74c554d713952d70e98ead WatchSource:0}: Error finding container 8a8ba8618bb4e7505ef39257fc27bd5b336831cb6c74c554d713952d70e98ead: Status 404 returned error can't find the container with id 8a8ba8618bb4e7505ef39257fc27bd5b336831cb6c74c554d713952d70e98ead
Oct 30 00:58:28.455864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.455814    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:28.455960 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.455927    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:28.456020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.455998    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:28.465364 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.465325    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:28.621147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.621116    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:28.776792 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:28.776761    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4c5fae_0415_46ea_bcdd_ba3713fc8ca4.slice/crio-83a12bf46c755b96b7fb48442c784774c720dead2813e475f51fbc25dfc0d501 WatchSource:0}: Error finding container 83a12bf46c755b96b7fb48442c784774c720dead2813e475f51fbc25dfc0d501: Status 404 returned error can't find the container with id 83a12bf46c755b96b7fb48442c784774c720dead2813e475f51fbc25dfc0d501
Oct 30 00:58:28.776984 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.776791    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"]
Oct 30 00:58:28.975024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.974984    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" event={"ID":"e0856659-4315-48f2-be0f-def689ca8af3","Type":"ContainerStarted","Data":"b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833"}
Oct 30 00:58:28.975448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.975051    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" event={"ID":"e0856659-4315-48f2-be0f-def689ca8af3","Type":"ContainerStarted","Data":"8a8ba8618bb4e7505ef39257fc27bd5b336831cb6c74c554d713952d70e98ead"}
Oct 30 00:58:28.976776 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:28.976750    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w" event={"ID":"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4","Type":"ContainerStarted","Data":"83a12bf46c755b96b7fb48442c784774c720dead2813e475f51fbc25dfc0d501"}
Oct 30 00:58:29.026341 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:29.026249    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podStartSLOduration=2.026236484 podStartE2EDuration="2.026236484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:29.023405424 +0000 UTC m=+1017.052715658" watchObservedRunningTime="2026-10-30 00:58:29.026236484 +0000 UTC m=+1017.055546715"
Oct 30 00:58:29.124448 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:29.124420    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:29.125645 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:29.125618    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:29.125788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:29.125667    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:29.986949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:29.986907    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w" event={"ID":"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4","Type":"ContainerStarted","Data":"7908f8a31d4028d92d55b5620db9eaefd083404cd06002d06408d7d12f999c76"}
Oct 30 00:58:30.031379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:30.031330    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w" podStartSLOduration=2.031316163 podStartE2EDuration="2.031316163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:30.03006188 +0000 UTC m=+1018.059372112" watchObservedRunningTime="2026-10-30 00:58:30.031316163 +0000 UTC m=+1018.060626394"
Oct 30 00:58:30.123763 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:30.123723    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:30.123985 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:30.123779    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:30.431520 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:30.431491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:30.995132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:30.995098    4577 generic.go:356] "Generic (PLEG): container finished" podID="af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" containerID="7908f8a31d4028d92d55b5620db9eaefd083404cd06002d06408d7d12f999c76" exitCode=0
Oct 30 00:58:30.995549 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:30.995162    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w" event={"ID":"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4","Type":"ContainerDied","Data":"7908f8a31d4028d92d55b5620db9eaefd083404cd06002d06408d7d12f999c76"}
Oct 30 00:58:31.123726 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:31.123698    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:31.123939 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:31.123753    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:31.212951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:31.212921    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:31.225788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:31.225761    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:32.014041 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.014012    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:32.124235 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.124202    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:32.124389 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.124281    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:32.224787 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.224762    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:32.367868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.362912    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:32.402547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.401167    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974\" (UniqueName: \"kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974\") pod \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") "
Oct 30 00:58:32.402547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.401232    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca\") pod \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") "
Oct 30 00:58:32.402547 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.401534    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca\") pod \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\" (UID: \"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4\") "
Oct 30 00:58:32.414970 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.414916    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca" pod "af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" (UID: "af4c5fae-0415-46ea-bcdd-ba3713fc8ca4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:58:32.420870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.420366    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974" pod "af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" (UID: "af4c5fae-0415-46ea-bcdd-ba3713fc8ca4"). InnerVolumeSpecName "kube-api-access-n4974". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:58:32.426401 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.426353    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca" pod "af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" (UID: "af4c5fae-0415-46ea-bcdd-ba3713fc8ca4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:58:32.502874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.502822    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:32.502874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.502874    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4974\" (UniqueName: \"kubernetes.io/projected/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-kube-api-access-n4974\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:32.503106 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:32.502888    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:33.010509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.010474    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"
Oct 30 00:58:33.010509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.010496    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w" event={"ID":"af4c5fae-0415-46ea-bcdd-ba3713fc8ca4","Type":"ContainerDied","Data":"83a12bf46c755b96b7fb48442c784774c720dead2813e475f51fbc25dfc0d501"}
Oct 30 00:58:33.010743 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.010551    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83a12bf46c755b96b7fb48442c784774c720dead2813e475f51fbc25dfc0d501"
Oct 30 00:58:33.123718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.123685    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:33.124210 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.123734    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:33.405873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.405755    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"]
Oct 30 00:58:33.406345 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.406325    4577 cpu_manager.go:414] "containerMap: removing container" podUID="af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" containerName="revoke-keys"
Oct 30 00:58:33.406345 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.406344    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" containerName="revoke-keys"
Oct 30 00:58:33.406506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.406447    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" containerName="revoke-keys"
Oct 30 00:58:33.414264 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.414244    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.434797 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.434767    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"]
Oct 30 00:58:33.514542 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514505    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.514717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514559    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.514717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514641    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.514814 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514748    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.514814 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514795    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.514926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514839    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.514990 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.514971    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.515036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.515006    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnmx4\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.515123 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.515068    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.616502 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616463    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.616670 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616518    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnmx4\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.616670 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616573    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.616670 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616630    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.616885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616674    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.616885 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616706    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.616929    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617065 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617008    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617124 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617062    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617124 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617081    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617212 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617163    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617316 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617297    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617388 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617372    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.617469 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.617450    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.619661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.619633    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.619789 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.619765    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.642198 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.642170    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.642749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.642726    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnmx4\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:33.728815 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:33.728784    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:34.083661 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.083629    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"]
Oct 30 00:58:34.084157 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:34.084128    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod753f43f0_4c9f_47a6_bf56_6a80eba29455.slice/crio-c03108d1e8447a6ef7137be6d78c647e8f75eb91888c5785d95f69a2d4642c5c WatchSource:0}: Error finding container c03108d1e8447a6ef7137be6d78c647e8f75eb91888c5785d95f69a2d4642c5c: Status 404 returned error can't find the container with id c03108d1e8447a6ef7137be6d78c647e8f75eb91888c5785d95f69a2d4642c5c
Oct 30 00:58:34.086629 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.086593    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:34.086716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.086659    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:34.086716 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.086690    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:34.124053 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.124027    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:34.124408 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.124075    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:34.238863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:34.238810    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:35.025932 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.025895    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" event={"ID":"753f43f0-4c9f-47a6-bf56-6a80eba29455","Type":"ContainerStarted","Data":"7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec"}
Oct 30 00:58:35.026129 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.025950    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" event={"ID":"753f43f0-4c9f-47a6-bf56-6a80eba29455","Type":"ContainerStarted","Data":"c03108d1e8447a6ef7137be6d78c647e8f75eb91888c5785d95f69a2d4642c5c"}
Oct 30 00:58:35.091484 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.091416    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podStartSLOduration=2.091395456 podStartE2EDuration="2.091395456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:35.08736218 +0000 UTC m=+1023.116672440" watchObservedRunningTime="2026-10-30 00:58:35.091395456 +0000 UTC m=+1023.120705687"
Oct 30 00:58:35.124759 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.124719    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 00:58:35.125263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.124785    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 00:58:35.525025 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.524992    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"]
Oct 30 00:58:35.529305 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.529276    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.567869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.567120    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"]
Oct 30 00:58:35.670869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670569    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.670869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670662    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.670869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670741    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.670869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670789    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.670869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670839    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbtft\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.671300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670900    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.671300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670946    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.671300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.670990    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.671300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.671030    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.729639 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.729577    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:35.732344 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.732305    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 00:58:35.732518 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.732371    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 00:58:35.783597 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783493    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783597 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783567    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbtft\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783603    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783648    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783687    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783720    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783793    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.783837 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783870    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.784201 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.783943    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.784649 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.784619    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.785389 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.785362    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.785650 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.785628    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.785907 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.785879    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.786249 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.786231    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.789983 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.789924    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.790509 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.790491    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.812383 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.812309    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbtft\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.812383 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.812323    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:35.853874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:35.844887    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:36.021100 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.021064    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:36.241358 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.241310    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"]
Oct 30 00:58:36.302896 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.267860    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:36.302896 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.267957    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:36.302896 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.267998    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:36.456182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.454467    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:36.457148 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.456675    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:36.489216 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.489182    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 00:58:36.729599 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.729564    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 00:58:36.729902 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:36.729614    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 00:58:37.041224 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.041176    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" event={"ID":"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c","Type":"ContainerStarted","Data":"dfdefb346870153b888521158e493fec84db4da5226642ab7427fc0bf749b3ed"}
Oct 30 00:58:37.041432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.041274    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" event={"ID":"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c","Type":"ContainerStarted","Data":"4ca9d5632dd1d6a26a31ae9b3f962a755ecf20a7f9a5f4052a09d21f7b4f0e56"}
Oct 30 00:58:37.098826 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.098776    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podStartSLOduration=2.098761243 podStartE2EDuration="2.098761243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:37.096475705 +0000 UTC m=+1025.125785936" watchObservedRunningTime="2026-10-30 00:58:37.098761243 +0000 UTC m=+1025.128071474"
Oct 30 00:58:37.710434 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.710400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:37.729168 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.729142    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 00:58:37.729381 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.729359    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 00:58:37.856867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.847800    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:37.864869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.858046    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:37.864869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:37.858106    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:38.729380 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:38.729331    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 00:58:38.729785 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:38.729408    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 00:58:38.849892 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:38.849169    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:38.849892 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:38.849228    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:39.601868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:39.601414    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:39.730298 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:39.730259    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 00:58:39.730959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:39.730925    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 00:58:39.849153 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:39.849119    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:39.849427 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:39.849402    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:40.729786 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:40.729746    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 00:58:40.729964 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:40.729828    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 00:58:40.858869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:40.849141    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:40.858869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:40.849200    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:41.416142 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:41.416101    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:41.857873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:41.854181    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:41.857873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:41.854245    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:41.919208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:41.919157    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:41.920233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:41.920141    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:41.921056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:41.921016    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 00:58:42.027223 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.027186    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:42.276979 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.276931    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"]
Oct 30 00:58:42.384868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.384686    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"]
Oct 30 00:58:42.384868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.384762    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.402055 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.402030    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:58:42.402290 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.402063    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:58:42.485662 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.485624    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q57tx\" (UniqueName: \"kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.485935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.485913    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.485985 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.485968    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.587877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.587575    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.587877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.587630    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.587877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.587709    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q57tx\" (UniqueName: \"kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.589316 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.589287    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.591992 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.590777    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.616354 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.616320    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q57tx\" (UniqueName: \"kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx\") pod \"maas-api-revoke-sub-fd554126-p8dnc\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.703952 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.703913    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:42.888530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.882066    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:42.888530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:42.882123    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:43.261086 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:43.244273    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"]
Oct 30 00:58:43.291732 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:43.291696    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc" event={"ID":"8f38c861-1a2a-48a8-85dc-698fc63c8897","Type":"ContainerStarted","Data":"c300ef141ada3caa598607fff08e0a418ae3b5adab6fbbdf15508f3f65a27c2f"}
Oct 30 00:58:43.339871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:43.333702    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:43.858925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:43.846214    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:43.858925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:43.846272    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:44.332795 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:44.332704    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc" event={"ID":"8f38c861-1a2a-48a8-85dc-698fc63c8897","Type":"ContainerStarted","Data":"44dcdd465ac6e1b4055e155097f4d30bceebd2a00c32de8f92640b82ad5c77d8"}
Oct 30 00:58:44.388873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:44.388138    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc" podStartSLOduration=2.388115908 podStartE2EDuration="2.388115908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:44.377301787 +0000 UTC m=+1032.406612029" watchObservedRunningTime="2026-10-30 00:58:44.388115908 +0000 UTC m=+1032.417426168"
Oct 30 00:58:44.849884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:44.848196    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 00:58:44.849884 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:44.848256    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 00:58:45.011873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.011321    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:58:45.185996 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.185946    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:45.252997 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.252947    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:45.508933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.487644    4577 generic.go:356] "Generic (PLEG): container finished" podID="8f38c861-1a2a-48a8-85dc-698fc63c8897" containerID="44dcdd465ac6e1b4055e155097f4d30bceebd2a00c32de8f92640b82ad5c77d8" exitCode=0
Oct 30 00:58:45.508933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.488016    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc" event={"ID":"8f38c861-1a2a-48a8-85dc-698fc63c8897","Type":"ContainerDied","Data":"44dcdd465ac6e1b4055e155097f4d30bceebd2a00c32de8f92640b82ad5c77d8"}
Oct 30 00:58:45.735118 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.735049    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:58:45.735389 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.735358    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:45.758284 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.758241    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 00:58:45.806657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:45.806565    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:46.014006 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.013954    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:58:46.015679 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.015654    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:46.294015 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.269682    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:46.294015 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.270046    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:58:46.294015 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.271820    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:46.387915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.360546    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:58:46.814493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.788488    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:46.814493 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.814217    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:46.913915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.888375    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:58:46.913915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.888444    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:58:46.932925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.932776    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:58:46.933327 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.933278    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:46.937147 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:46.937116    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/health\": context deadline exceeded"
Oct 30 00:58:47.283353 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.283296    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 00:58:47.284421 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.284398    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:47.290085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.290065    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:47.310198 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.310168    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 00:58:47.336533 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.336505    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:58:47.429510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.429473    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:58:47.527167 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.527131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:47.709667 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.709525    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:47.759676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.756737    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"
Oct 30 00:58:47.759676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.756942    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc" event={"ID":"8f38c861-1a2a-48a8-85dc-698fc63c8897","Type":"ContainerDied","Data":"c300ef141ada3caa598607fff08e0a418ae3b5adab6fbbdf15508f3f65a27c2f"}
Oct 30 00:58:47.759676 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.756983    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c300ef141ada3caa598607fff08e0a418ae3b5adab6fbbdf15508f3f65a27c2f"
Oct 30 00:58:47.810367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.810335    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx\" (UniqueName: \"kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx\") pod \"8f38c861-1a2a-48a8-85dc-698fc63c8897\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") "
Oct 30 00:58:47.810543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.810406    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca\") pod \"8f38c861-1a2a-48a8-85dc-698fc63c8897\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") "
Oct 30 00:58:47.810627 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.810563    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca\") pod \"8f38c861-1a2a-48a8-85dc-698fc63c8897\" (UID: \"8f38c861-1a2a-48a8-85dc-698fc63c8897\") "
Oct 30 00:58:47.810976 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.810946    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca" pod "8f38c861-1a2a-48a8-85dc-698fc63c8897" (UID: "8f38c861-1a2a-48a8-85dc-698fc63c8897"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:58:47.811372 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.811353    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:47.813136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.813113    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx" pod "8f38c861-1a2a-48a8-85dc-698fc63c8897" (UID: "8f38c861-1a2a-48a8-85dc-698fc63c8897"). InnerVolumeSpecName "kube-api-access-q57tx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:58:47.813276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.813254    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca" pod "8f38c861-1a2a-48a8-85dc-698fc63c8897" (UID: "8f38c861-1a2a-48a8-85dc-698fc63c8897"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:58:47.913085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.912990    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q57tx\" (UniqueName: \"kubernetes.io/projected/8f38c861-1a2a-48a8-85dc-698fc63c8897-kube-api-access-q57tx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:47.913085 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:47.913020    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8f38c861-1a2a-48a8-85dc-698fc63c8897-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:48.485379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.485336    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"]
Oct 30 00:58:48.485999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.485980    4577 cpu_manager.go:414] "containerMap: removing container" podUID="8f38c861-1a2a-48a8-85dc-698fc63c8897" containerName="revoke-keys"
Oct 30 00:58:48.486111 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.486001    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f38c861-1a2a-48a8-85dc-698fc63c8897" containerName="revoke-keys"
Oct 30 00:58:48.486166 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.486141    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f38c861-1a2a-48a8-85dc-698fc63c8897" containerName="revoke-keys"
Oct 30 00:58:48.489688 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.489666    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.545641 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.545611    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"]
Oct 30 00:58:48.625790 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.625754    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.625790 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.625792    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626078 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.625819    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626078 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.626056    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626175 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.626103    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626336 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.626313    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626401 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.626386    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626454 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.626439    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsbj4\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.626535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.626509    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727426    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727647 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727479    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727647 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727605    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsbj4\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727778 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727664    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727857    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727909    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727918    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.727999 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.727952    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.728179 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.728058    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.728179 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.728100    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.728179 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.728150    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.728313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.728210    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.728313 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.728230    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.728424 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.728402    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.730498 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.730475    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.730735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.730711    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.753002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.752946    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.753420 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.753400    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsbj4\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4\") pod \"e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") " pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:48.812117 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:48.812080    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:49.140267 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.140242    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"]
Oct 30 00:58:49.140955 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:49.140927    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ff1993_06bf_48b1_b9a0_05dc8b9b1be4.slice/crio-ef37d9b6015d85ba3283c270cce8e57d902efbf76837ee8d5ab5e9bdd98bffff WatchSource:0}: Error finding container ef37d9b6015d85ba3283c270cce8e57d902efbf76837ee8d5ab5e9bdd98bffff: Status 404 returned error can't find the container with id ef37d9b6015d85ba3283c270cce8e57d902efbf76837ee8d5ab5e9bdd98bffff
Oct 30 00:58:49.143382 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.143355    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:49.143464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.143412    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:49.143464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.143438    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:49.319829 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.319796    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:49.780385 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.780351    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" event={"ID":"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4","Type":"ContainerStarted","Data":"e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff"}
Oct 30 00:58:49.780556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.780403    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" event={"ID":"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4","Type":"ContainerStarted","Data":"ef37d9b6015d85ba3283c270cce8e57d902efbf76837ee8d5ab5e9bdd98bffff"}
Oct 30 00:58:49.812478 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.812454    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:49.859947 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:49.859889    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podStartSLOduration=1.85986819 podStartE2EDuration="1.85986819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:49.841761572 +0000 UTC m=+1037.871071829" watchObservedRunningTime="2026-10-30 00:58:49.85986819 +0000 UTC m=+1037.889178421"
Oct 30 00:58:50.813832 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:50.813801    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 00:58:50.814219 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:50.813878    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 00:58:51.128321 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.128248    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"]
Oct 30 00:58:51.132334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.132317    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.173226 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.173198    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:58:51.173495 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.173477    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:58:51.173662 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.173646    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"]
Oct 30 00:58:51.231368 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.231336    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:51.292529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.292493    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddxwg\" (UniqueName: \"kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.313281 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.313249    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.313616 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.313597    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.417935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.415173    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.417935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.415242    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.417935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.415439    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddxwg\" (UniqueName: \"kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.417935 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.417001    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.426252 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.426225    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.440647 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.440621    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddxwg\" (UniqueName: \"kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg\") pod \"maas-api-revoke-sub-e90116f8-6zq49\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.455208 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.454032    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:51.776931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.776899    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"]
Oct 30 00:58:51.779597 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:51.779566    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc9ef92_672b_4f84_b0fa_0b15a86bcf20.slice/crio-41f8c634fd74f5bf7329859b9ee15a66aa0c39fde84a048b0d100ae25811b869 WatchSource:0}: Error finding container 41f8c634fd74f5bf7329859b9ee15a66aa0c39fde84a048b0d100ae25811b869: Status 404 returned error can't find the container with id 41f8c634fd74f5bf7329859b9ee15a66aa0c39fde84a048b0d100ae25811b869
Oct 30 00:58:51.803051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.802260    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49" event={"ID":"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20","Type":"ContainerStarted","Data":"41f8c634fd74f5bf7329859b9ee15a66aa0c39fde84a048b0d100ae25811b869"}
Oct 30 00:58:51.812501 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.812473    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 00:58:51.812596 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.812537    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 00:58:51.816484 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.816458    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"]
Oct 30 00:58:51.823585 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.822350    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.862590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.862084    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"]
Oct 30 00:58:51.922430 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922380    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922430 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922436    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922479    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922502    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922537    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922567    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922592    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922616    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:51.922717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:51.922652    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjww2\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023565 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023532    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023586    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023673    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023705    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023771    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023828    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.023909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023885    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023912    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.023963    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjww2\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.024025    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024069 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.024029    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024526 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.024496    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.024549    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.024653 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.024620    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.026206 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.026165    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.026318 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.026300    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.070544 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.070514    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.072162 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.072127    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjww2\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.193590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.193563    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:52.579256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.579054    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"]
Oct 30 00:58:52.580932 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:52.579781    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5f4dc1_6714_48ac_897a_faa1cc6e29e7.slice/crio-2c96986c87b07117ea22923df46f31b26ec1fb36b8890b268b7858cf45a4b68e WatchSource:0}: Error finding container 2c96986c87b07117ea22923df46f31b26ec1fb36b8890b268b7858cf45a4b68e: Status 404 returned error can't find the container with id 2c96986c87b07117ea22923df46f31b26ec1fb36b8890b268b7858cf45a4b68e
Oct 30 00:58:52.583124 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.583083    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:52.583218 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.583177    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:52.583280 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.583226    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:58:52.812294 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.812262    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" event={"ID":"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7","Type":"ContainerStarted","Data":"5bde485049c7d17a594d9403aa7622a5ee938a2417540eb0d40af1176482ca41"}
Oct 30 00:58:52.812557 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.812537    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" event={"ID":"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7","Type":"ContainerStarted","Data":"2c96986c87b07117ea22923df46f31b26ec1fb36b8890b268b7858cf45a4b68e"}
Oct 30 00:58:52.813045 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.813025    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 00:58:52.813191 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.813169    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 00:58:52.814136 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.814104    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49" event={"ID":"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20","Type":"ContainerStarted","Data":"f2e0f503ab8a24d3597c27e61b6ac49aab992d5a3bbcfcf3263df1c9d75ba621"}
Oct 30 00:58:52.883554 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:52.883428    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podStartSLOduration=1.883408159 podStartE2EDuration="1.883408159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:52.866299619 +0000 UTC m=+1040.895609851" watchObservedRunningTime="2026-10-30 00:58:52.883408159 +0000 UTC m=+1040.912718391"
Oct 30 00:58:53.072868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.071357    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:53.200924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.198935    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:58:53.200924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.199062    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:53.200924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.199110    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:53.812736 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.812692    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 00:58:53.812956 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.812745    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 00:58:53.825436 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.825354    4577 generic.go:356] "Generic (PLEG): container finished" podID="2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" containerID="f2e0f503ab8a24d3597c27e61b6ac49aab992d5a3bbcfcf3263df1c9d75ba621" exitCode=0
Oct 30 00:58:53.825594 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:53.825492    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49" event={"ID":"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20","Type":"ContainerDied","Data":"f2e0f503ab8a24d3597c27e61b6ac49aab992d5a3bbcfcf3263df1c9d75ba621"}
Oct 30 00:58:54.196931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:54.196292    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:54.196931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:54.196354    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:54.813371 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:54.813330    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 00:58:54.813551 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:54.813398    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 00:58:55.176936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.173584    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:55.234868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.209353    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:55.270992 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.270943    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:55.403378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.403350    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:55.490026 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.489392    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca\") pod \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") "
Oct 30 00:58:55.490026 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.489475    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg\" (UniqueName: \"kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg\") pod \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") "
Oct 30 00:58:55.490026 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.489511    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca\") pod \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\" (UID: \"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20\") "
Oct 30 00:58:55.490026 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.489996    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca" pod "2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" (UID: "2fc9ef92-672b-4f84-b0fa-0b15a86bcf20"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:58:55.498872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.498008    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg" pod "2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" (UID: "2fc9ef92-672b-4f84-b0fa-0b15a86bcf20"). InnerVolumeSpecName "kube-api-access-ddxwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:58:55.501683 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.501661    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca" pod "2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" (UID: "2fc9ef92-672b-4f84-b0fa-0b15a86bcf20"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:58:55.591810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.591764    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:55.591810 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.591804    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddxwg\" (UniqueName: \"kubernetes.io/projected/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-kube-api-access-ddxwg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:55.592024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.591818    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:58:55.812684 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.812597    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 00:58:55.812831 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.812690    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 00:58:55.841917 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.841889    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"
Oct 30 00:58:55.842114 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.841886    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49" event={"ID":"2fc9ef92-672b-4f84-b0fa-0b15a86bcf20","Type":"ContainerDied","Data":"41f8c634fd74f5bf7329859b9ee15a66aa0c39fde84a048b0d100ae25811b869"}
Oct 30 00:58:55.842114 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.842053    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41f8c634fd74f5bf7329859b9ee15a66aa0c39fde84a048b0d100ae25811b869"
Oct 30 00:58:55.977870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.977396    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"]
Oct 30 00:58:55.985230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.978253    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" containerName="revoke-keys"
Oct 30 00:58:55.985230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.978276    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" containerName="revoke-keys"
Oct 30 00:58:55.985230 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.978478    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" containerName="revoke-keys"
Oct 30 00:58:55.997506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:55.994443    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.003182 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.003158    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"]
Oct 30 00:58:56.012412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.012381    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:58:56.012538 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.012462    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:58:56.097729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.097633    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.097905 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.097767    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9htbp\" (UniqueName: \"kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.097905 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.097869    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.194295 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.194253    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:56.194449 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.194326    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:56.199010 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.198981    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.199158 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.199139    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.199254 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.199238    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9htbp\" (UniqueName: \"kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.199783 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.199756    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.202194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.202171    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.224261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.224232    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9htbp\" (UniqueName: \"kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp\") pod \"maas-api-revoke-sub-557d1ba7-s7wt6\" (UID: \"5d536388-1129-4593-b786-e18909331475\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.316756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.316676    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:58:56.688924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.688661    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"]
Oct 30 00:58:56.698685 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:58:56.698626    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d536388_1129_4593_b786_e18909331475.slice/crio-c7885133fdc192eb4a1d804c2c0f06232f20a0ca1cd11ca2e2e09cf9d14b47fb WatchSource:0}: Error finding container c7885133fdc192eb4a1d804c2c0f06232f20a0ca1cd11ca2e2e09cf9d14b47fb: Status 404 returned error can't find the container with id c7885133fdc192eb4a1d804c2c0f06232f20a0ca1cd11ca2e2e09cf9d14b47fb
Oct 30 00:58:56.878882 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.877406    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6" event={"ID":"5d536388-1129-4593-b786-e18909331475","Type":"ContainerStarted","Data":"c7885133fdc192eb4a1d804c2c0f06232f20a0ca1cd11ca2e2e09cf9d14b47fb"}
Oct 30 00:58:56.980038 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:56.974188    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:57.115897 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:58:57.113525    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:58:57.173880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.169204    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:57.173880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.170806    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:57.189117 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.188343    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 00:58:57.194356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.194245    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:57.194356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.194302    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:57.894381 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.891877    4577 generic.go:356] "Generic (PLEG): container finished" podID="5d536388-1129-4593-b786-e18909331475" containerID="a99c541a43e06e34940f6698d30308d169002f08526b4185a4713c381c5b41c6" exitCode=0
Oct 30 00:58:57.894381 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:57.892957    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6" event={"ID":"5d536388-1129-4593-b786-e18909331475","Type":"ContainerDied","Data":"a99c541a43e06e34940f6698d30308d169002f08526b4185a4713c381c5b41c6"}
Oct 30 00:58:58.194863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:58.194769    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:58.194863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:58.194822    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:58.723441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:58.723401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:58:59.197261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:59.197225    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:58:59.197819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:59.197784    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:58:59.844698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:58:59.844665    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:59:00.098869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.081228    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca\") pod \"5d536388-1129-4593-b786-e18909331475\" (UID: \"5d536388-1129-4593-b786-e18909331475\") "
Oct 30 00:59:00.098869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.081514    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp\" (UniqueName: \"kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp\") pod \"5d536388-1129-4593-b786-e18909331475\" (UID: \"5d536388-1129-4593-b786-e18909331475\") "
Oct 30 00:59:00.098869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.081675    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca\") pod \"5d536388-1129-4593-b786-e18909331475\" (UID: \"5d536388-1129-4593-b786-e18909331475\") "
Oct 30 00:59:00.099623 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.099591    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca" pod "5d536388-1129-4593-b786-e18909331475" (UID: "5d536388-1129-4593-b786-e18909331475"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.179464    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.180183    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp" pod "5d536388-1129-4593-b786-e18909331475" (UID: "5d536388-1129-4593-b786-e18909331475"). InnerVolumeSpecName "kube-api-access-9htbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.180822    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca" pod "5d536388-1129-4593-b786-e18909331475" (UID: "5d536388-1129-4593-b786-e18909331475"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.184048    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5d536388-1129-4593-b786-e18909331475-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.184076    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9htbp\" (UniqueName: \"kubernetes.io/projected/5d536388-1129-4593-b786-e18909331475-kube-api-access-9htbp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.184110    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5d536388-1129-4593-b786-e18909331475-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.195102    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:59:00.208870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.195166    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:59:00.257870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.247505    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6" event={"ID":"5d536388-1129-4593-b786-e18909331475","Type":"ContainerDied","Data":"c7885133fdc192eb4a1d804c2c0f06232f20a0ca1cd11ca2e2e09cf9d14b47fb"}
Oct 30 00:59:00.257870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.247587    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7885133fdc192eb4a1d804c2c0f06232f20a0ca1cd11ca2e2e09cf9d14b47fb"
Oct 30 00:59:00.867984 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.866961    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:00.897922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.896801    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:00.953217 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:00.953180    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:01.208724 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.208630    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 30 00:59:01.209013 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.208982    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 30 00:59:01.311871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.311278    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 00:59:01.435108 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.433929    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 00:59:01.621379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.621313    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"]
Oct 30 00:59:01.625871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.622346    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5d536388-1129-4593-b786-e18909331475" containerName="revoke-keys"
Oct 30 00:59:01.625871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.622369    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d536388-1129-4593-b786-e18909331475" containerName="revoke-keys"
Oct 30 00:59:01.625871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.622643    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d536388-1129-4593-b786-e18909331475" containerName="revoke-keys"
Oct 30 00:59:01.660543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.656539    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"]
Oct 30 00:59:01.660543 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.656673    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:01.681930 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.681147    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:59:01.681930 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.681519    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:59:01.884941 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.852588    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:01.914272 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.913996    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:01.915170 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.915136    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms48x\" (UniqueName: \"kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:01.929926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.870911    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:01.931471 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.912967    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:01.938433 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.938411    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:01.938701 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.938686    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:01.964054 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:01.964004    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:02.043626 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.043585    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.075993 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.075960    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms48x\" (UniqueName: \"kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.076366 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.076344    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.324915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.077247    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.324915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.101742    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:02.324915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.190569    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.324915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.223922    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:02.324915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.302203    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms48x\" (UniqueName: \"kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x\") pod \"maas-api-revoke-sub-ba393b1f-l9t7r\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.324915 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.307560    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:02.727397 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.727360    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:02.939918 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.887291    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:02.968922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.965526    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:59:02.968922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.966045    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:59:02.984715 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.984678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:02.985329 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:02.984966    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 00:59:03.205889 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:03.204581    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"]
Oct 30 00:59:03.205889 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:03.204835    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a26825c_3de9_4ff9_a705_5c0b4f4ad4bf.slice/crio-6133458acd7ba573127860e7a755167363e5f4c82cd53ef25e454e68bcbfe1c6 WatchSource:0}: Error finding container 6133458acd7ba573127860e7a755167363e5f4c82cd53ef25e454e68bcbfe1c6: Status 404 returned error can't find the container with id 6133458acd7ba573127860e7a755167363e5f4c82cd53ef25e454e68bcbfe1c6
Oct 30 00:59:03.888863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:03.888808    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r" event={"ID":"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf","Type":"ContainerStarted","Data":"bf2bd8ee651bd7b69b742d9a2130b4284082c61168c815fc89c95466723b65fc"}
Oct 30 00:59:03.889374 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:03.888907    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r" event={"ID":"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf","Type":"ContainerStarted","Data":"6133458acd7ba573127860e7a755167363e5f4c82cd53ef25e454e68bcbfe1c6"}
Oct 30 00:59:03.937455 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:03.937415    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:03.940787 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:03.940747    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r" podStartSLOduration=2.940735068 podStartE2EDuration="2.940735068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:03.93694808 +0000 UTC m=+1051.966258314" watchObservedRunningTime="2026-10-30 00:59:03.940735068 +0000 UTC m=+1051.970045299"
Oct 30 00:59:04.051863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:04.051812    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:04.609059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:04.609021    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:04.898801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:04.898716    4577 generic.go:356] "Generic (PLEG): container finished" podID="5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" containerID="bf2bd8ee651bd7b69b742d9a2130b4284082c61168c815fc89c95466723b65fc" exitCode=0
Oct 30 00:59:04.899186 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:04.898805    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r" event={"ID":"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf","Type":"ContainerDied","Data":"bf2bd8ee651bd7b69b742d9a2130b4284082c61168c815fc89c95466723b65fc"}
Oct 30 00:59:06.068830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.068805    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:06.138619 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.138583    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x\" (UniqueName: \"kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x\") pod \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") "
Oct 30 00:59:06.138805 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.138726    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca\") pod \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") "
Oct 30 00:59:06.138899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.138802    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca\") pod \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\" (UID: \"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf\") "
Oct 30 00:59:06.139237 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.139205    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca" pod "5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" (UID: "5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:06.139482 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.139457    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:06.140924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.140897    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x" pod "5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" (UID: "5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf"). InnerVolumeSpecName "kube-api-access-ms48x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:06.141374 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.141353    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca" pod "5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" (UID: "5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:06.240856 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.240807    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:06.240856 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.240860    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms48x\" (UniqueName: \"kubernetes.io/projected/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf-kube-api-access-ms48x\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:06.315673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.315643    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"]
Oct 30 00:59:06.316256 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.316241    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" containerName="revoke-keys"
Oct 30 00:59:06.316314 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.316257    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" containerName="revoke-keys"
Oct 30 00:59:06.316350 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.316332    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" containerName="revoke-keys"
Oct 30 00:59:06.319920 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.319902    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.338472 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.338445    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"]
Oct 30 00:59:06.409598 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.409566    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:06.442604 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.442575    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.442770 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.442630    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg9c5\" (UniqueName: \"kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.442875 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.442822    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.544428 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.544338    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.544601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.544459    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.544601 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.544505    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg9c5\" (UniqueName: \"kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.545043 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.545019    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.547016 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.546993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.570233 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.570209    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg9c5\" (UniqueName: \"kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5\") pod \"maas-api-revoke-sub-767c4416-65mwc\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.634516 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.634486    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:06.915546 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.915467    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r" event={"ID":"5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf","Type":"ContainerDied","Data":"6133458acd7ba573127860e7a755167363e5f4c82cd53ef25e454e68bcbfe1c6"}
Oct 30 00:59:06.915546 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.915510    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"
Oct 30 00:59:06.915782 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.915514    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6133458acd7ba573127860e7a755167363e5f4c82cd53ef25e454e68bcbfe1c6"
Oct 30 00:59:06.936696 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:06.936672    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"]
Oct 30 00:59:06.937712 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:06.937685    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3887e40d_c73d_4337_aab8_3b76cfb90bad.slice/crio-5f33c8c16e9e5e412692cc4a9a7f507de303e662fd4853fbecc043d9be9bfc5f WatchSource:0}: Error finding container 5f33c8c16e9e5e412692cc4a9a7f507de303e662fd4853fbecc043d9be9bfc5f: Status 404 returned error can't find the container with id 5f33c8c16e9e5e412692cc4a9a7f507de303e662fd4853fbecc043d9be9bfc5f
Oct 30 00:59:07.924445 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:07.924408    4577 generic.go:356] "Generic (PLEG): container finished" podID="3887e40d-c73d-4337-aab8-3b76cfb90bad" containerID="2a248348966459fa1ecce4618282aa3295c8f1dce4190cfe5250e1805f48efb7" exitCode=0
Oct 30 00:59:07.924834 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:07.924500    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc" event={"ID":"3887e40d-c73d-4337-aab8-3b76cfb90bad","Type":"ContainerDied","Data":"2a248348966459fa1ecce4618282aa3295c8f1dce4190cfe5250e1805f48efb7"}
Oct 30 00:59:07.924834 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:07.924552    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc" event={"ID":"3887e40d-c73d-4337-aab8-3b76cfb90bad","Type":"ContainerStarted","Data":"5f33c8c16e9e5e412692cc4a9a7f507de303e662fd4853fbecc043d9be9bfc5f"}
Oct 30 00:59:08.214858 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:08.214753    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:09.092596 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.092574    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:09.185797 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.185707    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca\") pod \"3887e40d-c73d-4337-aab8-3b76cfb90bad\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") "
Oct 30 00:59:09.185996 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.185873    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5\" (UniqueName: \"kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5\") pod \"3887e40d-c73d-4337-aab8-3b76cfb90bad\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") "
Oct 30 00:59:09.185996 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.185960    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca\") pod \"3887e40d-c73d-4337-aab8-3b76cfb90bad\" (UID: \"3887e40d-c73d-4337-aab8-3b76cfb90bad\") "
Oct 30 00:59:09.186292 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.186270    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca" pod "3887e40d-c73d-4337-aab8-3b76cfb90bad" (UID: "3887e40d-c73d-4337-aab8-3b76cfb90bad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:09.186652 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.186632    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:09.188041 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.188017    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca" pod "3887e40d-c73d-4337-aab8-3b76cfb90bad" (UID: "3887e40d-c73d-4337-aab8-3b76cfb90bad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:09.188216 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.188200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5" pod "3887e40d-c73d-4337-aab8-3b76cfb90bad" (UID: "3887e40d-c73d-4337-aab8-3b76cfb90bad"). InnerVolumeSpecName "kube-api-access-tg9c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:09.288020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.287992    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg9c5\" (UniqueName: \"kubernetes.io/projected/3887e40d-c73d-4337-aab8-3b76cfb90bad-kube-api-access-tg9c5\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:09.288020 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.288018    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3887e40d-c73d-4337-aab8-3b76cfb90bad-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:09.939352 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.939321    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc" event={"ID":"3887e40d-c73d-4337-aab8-3b76cfb90bad","Type":"ContainerDied","Data":"5f33c8c16e9e5e412692cc4a9a7f507de303e662fd4853fbecc043d9be9bfc5f"}
Oct 30 00:59:09.939530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.939361    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"
Oct 30 00:59:09.939530 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:09.939367    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f33c8c16e9e5e412692cc4a9a7f507de303e662fd4853fbecc043d9be9bfc5f"
Oct 30 00:59:10.137731 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.137463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:10.166871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.162960    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"]
Oct 30 00:59:10.179378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.174762    4577 cpu_manager.go:414] "containerMap: removing container" podUID="3887e40d-c73d-4337-aab8-3b76cfb90bad" containerName="revoke-keys"
Oct 30 00:59:10.179378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.174794    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3887e40d-c73d-4337-aab8-3b76cfb90bad" containerName="revoke-keys"
Oct 30 00:59:10.179378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.175248    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="3887e40d-c73d-4337-aab8-3b76cfb90bad" containerName="revoke-keys"
Oct 30 00:59:10.184466 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.184436    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"]
Oct 30 00:59:10.184760 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.184742    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.202407 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.202383    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:59:10.202665 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.202650    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:59:10.301022 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.300979    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.301204 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.301040    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvphl\" (UniqueName: \"kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.301300 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.301282    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.402780 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.402748    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.403003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.402791    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvphl\" (UniqueName: \"kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.403003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.402926    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.403476 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.403448    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.405424 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.405407    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.428527 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.428494    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvphl\" (UniqueName: \"kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl\") pod \"maas-api-revoke-sub-4972fecd-7k9qc\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.502055 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.502028    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:10.809323 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.809294    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"]
Oct 30 00:59:10.811491 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:10.811462    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b9f812_5e0e_4998_82eb_6b97a7a155a2.slice/crio-fb3c722b3d3bc272d927e463f3eac21e1b238dc624a06c4d448f050f2f4c5ae5 WatchSource:0}: Error finding container fb3c722b3d3bc272d927e463f3eac21e1b238dc624a06c4d448f050f2f4c5ae5: Status 404 returned error can't find the container with id fb3c722b3d3bc272d927e463f3eac21e1b238dc624a06c4d448f050f2f4c5ae5
Oct 30 00:59:10.948151 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:10.948107    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc" event={"ID":"a1b9f812-5e0e-4998-82eb-6b97a7a155a2","Type":"ContainerStarted","Data":"fb3c722b3d3bc272d927e463f3eac21e1b238dc624a06c4d448f050f2f4c5ae5"}
Oct 30 00:59:11.815206 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:11.815164    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:11.958500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:11.958408    4577 generic.go:356] "Generic (PLEG): container finished" podID="a1b9f812-5e0e-4998-82eb-6b97a7a155a2" containerID="2946467c91ce394084c65b7ea9cb462fb4fd1a9f4f2d17da7d33413aa1d9ec30" exitCode=0
Oct 30 00:59:11.958500 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:11.958480    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc" event={"ID":"a1b9f812-5e0e-4998-82eb-6b97a7a155a2","Type":"ContainerDied","Data":"2946467c91ce394084c65b7ea9cb462fb4fd1a9f4f2d17da7d33413aa1d9ec30"}
Oct 30 00:59:12.924859 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:12.924817    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"]
Oct 30 00:59:12.928793 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:12.928761    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:12.958317 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:12.958290    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"]
Oct 30 00:59:13.035688 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.035656    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.035881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.035707    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.035881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.035818    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.035981 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.035946    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.036026 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.036013    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvfrr\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.036161 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.036146    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.036260 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.036241    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.036327 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.036314    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.036374 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.036352    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.129974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.129951    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:13.137258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137238    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137280    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137334    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137384    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137424    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137483 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137468    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137533    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137588    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137647    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvfrr\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137729 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137700    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137859 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137757    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.137974 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137950    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.138042 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.137985    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.138165 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.138145    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.139772 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.139754    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.140018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.140002    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.162393 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.162367    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.162615 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.162596    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvfrr\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.238458 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.238427    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca\") pod \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") "
Oct 30 00:59:13.238673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.238505    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl\" (UniqueName: \"kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl\") pod \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") "
Oct 30 00:59:13.238673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.238562    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca\") pod \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\" (UID: \"a1b9f812-5e0e-4998-82eb-6b97a7a155a2\") "
Oct 30 00:59:13.238924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.238901    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca" pod "a1b9f812-5e0e-4998-82eb-6b97a7a155a2" (UID: "a1b9f812-5e0e-4998-82eb-6b97a7a155a2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:13.239325 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.239233    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:13.240753 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.240730    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca" pod "a1b9f812-5e0e-4998-82eb-6b97a7a155a2" (UID: "a1b9f812-5e0e-4998-82eb-6b97a7a155a2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:13.240872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.240833    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl" pod "a1b9f812-5e0e-4998-82eb-6b97a7a155a2" (UID: "a1b9f812-5e0e-4998-82eb-6b97a7a155a2"). InnerVolumeSpecName "kube-api-access-wvphl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:13.259334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.259275    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:13.341050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.341014    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:13.341050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.341053    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvphl\" (UniqueName: \"kubernetes.io/projected/a1b9f812-5e0e-4998-82eb-6b97a7a155a2-kube-api-access-wvphl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:13.504273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.504187    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:13.603914 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.602513    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"]
Oct 30 00:59:13.626996 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:13.626947    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc09a1df_6c62_4f2a_937f_c68a4c95852c.slice/crio-3dd93c8d203cd3e2cb13b9a0cc1a15c9258d48e972b540a2d3706c9ac3538ba6 WatchSource:0}: Error finding container 3dd93c8d203cd3e2cb13b9a0cc1a15c9258d48e972b540a2d3706c9ac3538ba6: Status 404 returned error can't find the container with id 3dd93c8d203cd3e2cb13b9a0cc1a15c9258d48e972b540a2d3706c9ac3538ba6
Oct 30 00:59:13.630228 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.630183    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:13.630351 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.630330    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:13.630440 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:13.630384    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:14.001135 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.001093    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" event={"ID":"cc09a1df-6c62-4f2a-937f-c68a4c95852c","Type":"ContainerStarted","Data":"20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7"}
Oct 30 00:59:14.001602 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.001209    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" event={"ID":"cc09a1df-6c62-4f2a-937f-c68a4c95852c","Type":"ContainerStarted","Data":"3dd93c8d203cd3e2cb13b9a0cc1a15c9258d48e972b540a2d3706c9ac3538ba6"}
Oct 30 00:59:14.003465 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.003433    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc" event={"ID":"a1b9f812-5e0e-4998-82eb-6b97a7a155a2","Type":"ContainerDied","Data":"fb3c722b3d3bc272d927e463f3eac21e1b238dc624a06c4d448f050f2f4c5ae5"}
Oct 30 00:59:14.003593 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.003484    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb3c722b3d3bc272d927e463f3eac21e1b238dc624a06c4d448f050f2f4c5ae5"
Oct 30 00:59:14.003593 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.003447    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"
Oct 30 00:59:14.052205 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.052148    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podStartSLOduration=2.052134117 podStartE2EDuration="2.052134117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:14.050369596 +0000 UTC m=+1062.079679849" watchObservedRunningTime="2026-10-30 00:59:14.052134117 +0000 UTC m=+1062.081444349"
Oct 30 00:59:14.259778 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.259698    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:14.261526 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.261505    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:14.261609 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:14.261552    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:15.265871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.260457    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:15.265871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.260541    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:15.328552 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.328519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:15.777242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.777210    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"]
Oct 30 00:59:15.779751 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.779730    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a1b9f812-5e0e-4998-82eb-6b97a7a155a2" containerName="revoke-keys"
Oct 30 00:59:15.780003 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.779988    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1b9f812-5e0e-4998-82eb-6b97a7a155a2" containerName="revoke-keys"
Oct 30 00:59:15.780367 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.780334    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1b9f812-5e0e-4998-82eb-6b97a7a155a2" containerName="revoke-keys"
Oct 30 00:59:15.787988 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.787965    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"]
Oct 30 00:59:15.788281 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.788219    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.809894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.809453    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:59:15.822306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.822285    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:59:15.894464 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.894431    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.894645 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.894509    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5cnn\" (UniqueName: \"kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.894687 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.894657    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.995690 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.995657    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5cnn\" (UniqueName: \"kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.995906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.995817    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.995982 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.995964    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.996534 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.996508    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:15.998711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:15.998684    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:16.021109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:16.021087    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5cnn\" (UniqueName: \"kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn\") pod \"maas-api-revoke-sub-c93eea16-f2xh7\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:16.106826 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:16.106744    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:16.279431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:16.279331    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:16.279431 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:16.279391    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:16.476576 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:16.476519    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"]
Oct 30 00:59:17.029054 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:17.029016    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7" event={"ID":"21b5a82e-4596-49f2-b487-a402602408d3","Type":"ContainerStarted","Data":"33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b"}
Oct 30 00:59:17.029202 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:17.029076    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7" event={"ID":"21b5a82e-4596-49f2-b487-a402602408d3","Type":"ContainerStarted","Data":"8368197dfb003e6345c1c9d4dd047811b1eea6fbe4f343a0b4559cf59cd852a6"}
Oct 30 00:59:17.081282 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:17.081214    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7" podStartSLOduration=2.081193418 podStartE2EDuration="2.081193418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:17.077813728 +0000 UTC m=+1065.107123959" watchObservedRunningTime="2026-10-30 00:59:17.081193418 +0000 UTC m=+1065.110503649"
Oct 30 00:59:17.265718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:17.265258    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:17.265718 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:17.265321    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:17.292889 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:17.284202    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:17.686412 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:17.683934    4577 helpers.go:246] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b5a82e_4596_49f2_b487_a402602408d3.slice/crio-33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b.scope/pids.max": read /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b5a82e_4596_49f2_b487_a402602408d3.slice/crio-33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b.scope/pids.max: no such device
Oct 30 00:59:17.960193 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:59:17.931067    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b5a82e_4596_49f2_b487_a402602408d3.slice/crio-conmon-33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 00:59:17.963968 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:59:17.935194    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b5a82e_4596_49f2_b487_a402602408d3.slice/crio-conmon-33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 00:59:18.179887 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:18.164910    4577 generic.go:356] "Generic (PLEG): container finished" podID="21b5a82e-4596-49f2-b487-a402602408d3" containerID="33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b" exitCode=0
Oct 30 00:59:18.317238 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:18.317196    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7" event={"ID":"21b5a82e-4596-49f2-b487-a402602408d3","Type":"ContainerDied","Data":"33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b"}
Oct 30 00:59:18.324876 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:18.318254    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:18.332416 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:18.332379    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:18.812178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:18.810996    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:19.125122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.125001    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:19.318872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.306165    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:19.318872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.306231    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:19.321511 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.321470    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:19.458874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.447734    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:19.834872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.830785    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:19.834872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.831490    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:19.975886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:19.964683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:20.211930 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.211087    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:20.274871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.273244    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:20.274871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.273306    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:20.327865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.322937    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:20.451067 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.438513    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:20.451067 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.438576    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:59:20.451067 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.450045    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:20.464306 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.462958    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 00:59:20.467869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.464536    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:59:20.834138 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:20.834091    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:21.655456 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:21.626984    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 00:59:21.720872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:21.713266    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:21.825384 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:21.812199    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:59:22.005836 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.005801    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:22.006171 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.006131    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:22.129163 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.129042    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:22.178868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.178186    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:22.178868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.178593    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:22.203871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.179106    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:22.358400 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.358346    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:22.358863 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.358804    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:22.712875 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.700143    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:22.740880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.739691    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:22.742294 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.742268    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:59:22.742510 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.742487    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7" event={"ID":"21b5a82e-4596-49f2-b487-a402602408d3","Type":"ContainerDied","Data":"8368197dfb003e6345c1c9d4dd047811b1eea6fbe4f343a0b4559cf59cd852a6"}
Oct 30 00:59:22.742717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.742685    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8368197dfb003e6345c1c9d4dd047811b1eea6fbe4f343a0b4559cf59cd852a6"
Oct 30 00:59:22.742856 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.742827    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"]
Oct 30 00:59:22.744739 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.744720    4577 cpu_manager.go:414] "containerMap: removing container" podUID="21b5a82e-4596-49f2-b487-a402602408d3" containerName="revoke-keys"
Oct 30 00:59:22.744912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.744896    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21b5a82e-4596-49f2-b487-a402602408d3" containerName="revoke-keys"
Oct 30 00:59:22.745537 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.745521    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="21b5a82e-4596-49f2-b487-a402602408d3" containerName="revoke-keys"
Oct 30 00:59:22.781586 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.781550    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:22.781895 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.781836    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:22.808038 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.792167    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"]
Oct 30 00:59:22.823872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.820155    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:22.863148 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.863118    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:22.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.874309    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca\") pod \"21b5a82e-4596-49f2-b487-a402602408d3\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") "
Oct 30 00:59:22.889998 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.886160    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn\" (UniqueName: \"kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn\") pod \"21b5a82e-4596-49f2-b487-a402602408d3\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") "
Oct 30 00:59:22.889998 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.886326    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca\") pod \"21b5a82e-4596-49f2-b487-a402602408d3\" (UID: \"21b5a82e-4596-49f2-b487-a402602408d3\") "
Oct 30 00:59:22.889998 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.882593    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca" pod "21b5a82e-4596-49f2-b487-a402602408d3" (UID: "21b5a82e-4596-49f2-b487-a402602408d3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:22.910871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.883798    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:22.946030 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.945978    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:22.998912 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.988599    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt27w\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.002871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:22.999265    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.006945 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.006912    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.013934 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.013903    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.017866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.014148    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.043257 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.043225    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dtb\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.043525 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.043502    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.050867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.046954    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.061964 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.061931    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.062766 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.062277    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.078033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.077998    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.078289 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.078267    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.078432 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.078418    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.083148 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.083108    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.083428 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.083406    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.087161 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.087138    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.092098 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.092071    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.106868 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.092340    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.124343 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.124312    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:23.124582 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.087014    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"]
Oct 30 00:59:23.243673 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.243576    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca" pod "21b5a82e-4596-49f2-b487-a402602408d3" (UID: "21b5a82e-4596-49f2-b487-a402602408d3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:23.247465 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.246116    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.250075    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn" pod "21b5a82e-4596-49f2-b487-a402602408d3" (UID: "21b5a82e-4596-49f2-b487-a402602408d3"). InnerVolumeSpecName "kube-api-access-g5cnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.250940    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt27w\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251034    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251060    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251083    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251124    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251157    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dtb\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251250    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251317    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251350    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251382    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251429    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251463    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251495    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251522    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251552    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251583    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251633    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251677    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251770    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5cnn\" (UniqueName: \"kubernetes.io/projected/21b5a82e-4596-49f2-b487-a402602408d3-kube-api-access-g5cnn\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:23.252513 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.251789    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/21b5a82e-4596-49f2-b487-a402602408d3-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:23.271506 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.271458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:23.301978 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.292537    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.301978 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.293512    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.327017 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.302399    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.327017 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.303050    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.439434 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.426093    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.439434 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.434243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:23.446785 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.445818    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt27w\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.527082 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.514022    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:23.527082 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.522974    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:23.550869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.545902    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.550869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.546191    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.550869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.546525    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.550869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.546748    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.567051 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.567007    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"]
Oct 30 00:59:23.628517 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.628484    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.637901 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.632601    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dtb\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.644981 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.639199    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.725612 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.702018    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 00:59:23.725612 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.702450    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 00:59:23.741928 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.730561    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.743032 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.742182    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:23.778335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.762959    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.778335 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.763471    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.779406 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.778920    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.790118 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.790085    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:23.830593 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.830562    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:23.852877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:23.845501    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"
Oct 30 00:59:24.054633 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.054304    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:24.252059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.252031    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:24.278378 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.278349    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:24.278586 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.278568    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"]
Oct 30 00:59:24.280005 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.260211    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:24.280203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.280180    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:24.288747 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.288724    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"]
Oct 30 00:59:24.289098 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.289003    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.310436 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.310357    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8363a40b-openshift-default-dockercfg-s5kzj\""
Oct 30 00:59:24.358589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.358552    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393386    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393534    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlglj\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393564    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393642    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393732    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393863    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393892    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.395874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.393984    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.436826 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.436800    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"]
Oct 30 00:59:24.441035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.440988    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:24.441166 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.441101    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:24.441225 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.441179    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.501831    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.501953    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.501981    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502046    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502101    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502137    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502184    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlglj\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502211    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502271    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.504222 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.502810    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.526949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.505796    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.526949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.506605    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.526949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.506881    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.526949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.507392    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.526949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.507903    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.535892 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.533063    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.540879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.538783    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.540879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.539689    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlglj\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj\") pod \"e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") " pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.637665 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.637631    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:24.682383 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.680149    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"]
Oct 30 00:59:24.687309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.687148    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:24.687309 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.687269    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:24.687508 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.687342    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:24.722132 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.722101    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:24.906859 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.906101    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:24.916864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.915948    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" event={"ID":"489295d0-136f-47e6-9faf-4ebc614d6eb7","Type":"ContainerStarted","Data":"c444776022655decff217e1db0c5478fd0a65caabbdb79d1b1f77d9b41271494"}
Oct 30 00:59:24.916864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.916017    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" event={"ID":"489295d0-136f-47e6-9faf-4ebc614d6eb7","Type":"ContainerStarted","Data":"83133f4b1d80d47638cf33d3b5caf19a0c9b4499d168a2af7e8aa017a359d722"}
Oct 30 00:59:24.920639 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.919877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" event={"ID":"ec1fb88b-9674-4709-aca3-445a57945d83","Type":"ContainerStarted","Data":"1278349d97c1ec3f3c5dacc8df094efbfc2afee80b932dfcf5a0954299a3f6f9"}
Oct 30 00:59:24.920639 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.919932    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" event={"ID":"ec1fb88b-9674-4709-aca3-445a57945d83","Type":"ContainerStarted","Data":"524ed6d12c065c9b15edeb0289a5df9ec706b02e5019b23c641af78c1f34d32f"}
Oct 30 00:59:24.993525 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:24.989504    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podStartSLOduration=4.989483698 podStartE2EDuration="4.989483698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:24.975942483 +0000 UTC m=+1073.005252715" watchObservedRunningTime="2026-10-30 00:59:24.989483698 +0000 UTC m=+1073.018793928"
Oct 30 00:59:25.029891 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.029807    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podStartSLOduration=5.02978987 podStartE2EDuration="5.02978987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:25.02524886 +0000 UTC m=+1073.054559093" watchObservedRunningTime="2026-10-30 00:59:25.02978987 +0000 UTC m=+1073.059100105"
Oct 30 00:59:25.038479 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.038450    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"]
Oct 30 00:59:25.039708 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.039676    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:25.041590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.041552    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:25.041674 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.041661    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:25.041737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.041720    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:25.058263 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.058165    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:25.060536 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.060409    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:25.060536 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.060458    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:25.260529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.260499    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 00:59:25.260767 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.260741    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 00:59:25.458771 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.458732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:25.798865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.796249    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:25.798865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.798228    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:25.798865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.798278    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:25.991869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:25.989057    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:26.093229 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.072315    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"]
Oct 30 00:59:26.180764 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.138639    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:26.230047 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.230004    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:26.236016 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.235985    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:26.242523 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.242485    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" event={"ID":"fe8e2ee4-1059-413c-90d7-feaf02e70ed3","Type":"ContainerStarted","Data":"65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"}
Oct 30 00:59:26.242674 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.242531    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" event={"ID":"fe8e2ee4-1059-413c-90d7-feaf02e70ed3","Type":"ContainerStarted","Data":"f8d4fd10cc3244db42ba8bfc804c62a691dbb64f611495c6b031fc00e9b94765"}
Oct 30 00:59:26.242674 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.242561    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"]
Oct 30 00:59:26.242924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.242904    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.279970 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.279189    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:59:26.287575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.287548    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:59:26.425008 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.417878    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podStartSLOduration=3.417835051 podStartE2EDuration="3.417835051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:26.343323545 +0000 UTC m=+1074.372633777" watchObservedRunningTime="2026-10-30 00:59:26.417835051 +0000 UTC m=+1074.447145283"
Oct 30 00:59:26.437634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.437601    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.438492 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.438466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bwb5\" (UniqueName: \"kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.439079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.439058    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.493491 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.493296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:26.543074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.542423    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.543916 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.543462    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bwb5\" (UniqueName: \"kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.544106 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.544081    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.553138 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.553076    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.569379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.569320    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.574083 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.574011    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bwb5\" (UniqueName: \"kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5\") pod \"maas-api-revoke-sub-3c0179e3-725fk\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.629945 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.629915    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:26.638281 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.638200    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:26.664962 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.664578    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:26.795239 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.794763    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:26.795239 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:26.794823    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:27.096695 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.096241    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:27.096695 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.096303    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:27.135815 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:59:27.135782    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:59:27.236380 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.236353    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"]
Oct 30 00:59:27.239122 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:27.238555    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b2d8b5_fcd8_49b5_817f_7bdf2f32df6e.slice/crio-98be4d6f55916b86630ed88ef2e2993d683b1f770fc1c823cd1b2e69d4579f25 WatchSource:0}: Error finding container 98be4d6f55916b86630ed88ef2e2993d683b1f770fc1c823cd1b2e69d4579f25: Status 404 returned error can't find the container with id 98be4d6f55916b86630ed88ef2e2993d683b1f770fc1c823cd1b2e69d4579f25
Oct 30 00:59:27.320446 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.320410    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:27.321498 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.321464    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:27.325924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.325903    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk" event={"ID":"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e","Type":"ContainerStarted","Data":"98be4d6f55916b86630ed88ef2e2993d683b1f770fc1c823cd1b2e69d4579f25"}
Oct 30 00:59:27.326104 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.326092    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:27.327324 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.327299    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 00:59:27.762171 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.762137    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:27.796324 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.794941    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:27.796324 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:27.794999    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:28.062827 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.062780    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:28.063452 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.063420    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:28.268871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.259381    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:28.364880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.363937    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:28.364880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.364001    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:28.387870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.375132    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk" event={"ID":"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e","Type":"ContainerStarted","Data":"a3ab27521f2c41e240d622cb41f0e0610c8ac211b4c17972e4494e826df4723b"}
Oct 30 00:59:28.438873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.436351    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk" podStartSLOduration=3.43633245 podStartE2EDuration="3.43633245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:28.425546696 +0000 UTC m=+1076.454856933" watchObservedRunningTime="2026-10-30 00:59:28.43633245 +0000 UTC m=+1076.465642681"
Oct 30 00:59:28.734779 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.734694    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:28.738926 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.738897    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 00:59:28.799365 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.799329    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:28.799612 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.799586    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:28.939983 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:28.938713    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:29.071904 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.071791    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:29.078074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.072244    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:29.078074 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.072298    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:29.416002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.415967    4577 generic.go:356] "Generic (PLEG): container finished" podID="a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" containerID="a3ab27521f2c41e240d622cb41f0e0610c8ac211b4c17972e4494e826df4723b" exitCode=0
Oct 30 00:59:29.416647 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.416082    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk" event={"ID":"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e","Type":"ContainerDied","Data":"a3ab27521f2c41e240d622cb41f0e0610c8ac211b4c17972e4494e826df4723b"}
Oct 30 00:59:29.809899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.795226    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:29.809899 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.795291    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:29.975870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:29.974314    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:30.077924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.066282    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:30.077924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.066342    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:30.627419 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.627382    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:30.737017 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.736993    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:30.743118 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.743092    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"]
Oct 30 00:59:30.743691 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.743675    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" containerName="revoke-keys"
Oct 30 00:59:30.743691 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.743691    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" containerName="revoke-keys"
Oct 30 00:59:30.743864 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.743778    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" containerName="revoke-keys"
Oct 30 00:59:30.803871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.790905    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:30.808872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.807189    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:30.810866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.809336    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:30.810866 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.809392    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:30.838088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.822669    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"]
Oct 30 00:59:30.838088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.824620    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8363a40b\""
Oct 30 00:59:30.838088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.835889    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"]
Oct 30 00:59:30.841568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.841544    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:30.869496 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.843802    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8363a40b-dockercfg-cffx6\""
Oct 30 00:59:30.908805 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.908755    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca\") pod \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") "
Oct 30 00:59:30.909027 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.908979    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca\") pod \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") "
Oct 30 00:59:30.909151 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.909128    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5\" (UniqueName: \"kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5\") pod \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\" (UID: \"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e\") "
Oct 30 00:59:30.944590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.940500    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca" pod "a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" (UID: "a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:30.944590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.941602    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv2c2\" (UniqueName: \"kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2\") pod \"payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:30.944590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.941892    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:30.944590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.942042    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:30.949973 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:30.949910    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5" pod "a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" (UID: "a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e"). InnerVolumeSpecName "kube-api-access-4bwb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:31.024871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.010335    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca" pod "a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" (UID: "a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.042733    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume\") pod \"payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") " pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.042884    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.042998    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2c2\" (UniqueName: \"kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2\") pod \"payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.043051    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khb7q\" (UniqueName: \"kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q\") pod \"payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") " pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.043134    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bwb5\" (UniqueName: \"kubernetes.io/projected/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-kube-api-access-4bwb5\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.043150    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:31.044356 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.044034    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:31.057737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.057277    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:31.057737 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.057339    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:31.108947 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.108904    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"]
Oct 30 00:59:31.140871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.137537    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv2c2\" (UniqueName: \"kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2\") pod \"payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:31.168865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.162172    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khb7q\" (UniqueName: \"kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q\") pod \"payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") " pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.168865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.162273    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume\") pod \"payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") " pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.168865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.163199    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume\") pod \"payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") " pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.356876 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.285647    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khb7q\" (UniqueName: \"kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q\") pod \"payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") " pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.356876 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.297171    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:31.589871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.578273    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:31.826873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.805567    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:31.827542 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.827504    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:31.933059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.904964    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:31.979027 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:31.978983    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:32.014310 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.014257    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk" event={"ID":"a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e","Type":"ContainerDied","Data":"98be4d6f55916b86630ed88ef2e2993d683b1f770fc1c823cd1b2e69d4579f25"}
Oct 30 00:59:32.014634 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.014614    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98be4d6f55916b86630ed88ef2e2993d683b1f770fc1c823cd1b2e69d4579f25"
Oct 30 00:59:32.014900 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.014884    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"
Oct 30 00:59:32.063711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.063672    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:32.064042 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.064007    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:32.257933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.237249    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/health\": context deadline exceeded"
Oct 30 00:59:32.352901 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.335101    4577 scope.go:122] "RemoveContainer" containerID="2f04424a439af9623cb3b7e2ba45d17dcac5ceb195e8e2d7cac4abca6e606118"
Oct 30 00:59:32.634749 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.634717    4577 scope.go:122] "RemoveContainer" containerID="379777c68fd5cce0cee85fb2614168aa4c90d6a939e3efb5e91819dadf3784ca"
Oct 30 00:59:32.673872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.671208    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:32.699874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.689228    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:32.865370 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.865256    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:32.920867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.917713    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:32.954874 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.927273    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:32.961534 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.961462    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:33.024906 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:32.970934    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:33.245787 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.245334    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:33.245787 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.245404    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:33.336079 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.330315    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:33.894881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.675980    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:33.894881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.678366    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:59:33.894881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.717333    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 00:59:33.894881 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.718764    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:59:33.957873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.928477    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:33.957873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:33.928552    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:34.021873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.018236    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:34.173928 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.162152    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:34.206875 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.179238    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:34.207163 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.207128    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:34.570717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.568448    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:34.570717 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.568509    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:34.708869 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:34.703964    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e69b637_61db_439b_a9d5_5878afeed344.slice/crio-5318f531d89f9dbe5cf9d3fc486b430d716ff5f6e32a5c1f0d5c5ce43d3662b6 WatchSource:0}: Error finding container 5318f531d89f9dbe5cf9d3fc486b430d716ff5f6e32a5c1f0d5c5ce43d3662b6: Status 404 returned error can't find the container with id 5318f531d89f9dbe5cf9d3fc486b430d716ff5f6e32a5c1f0d5c5ce43d3662b6
Oct 30 00:59:34.945160 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.942505    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 00:59:34.945160 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:34.942567    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 00:59:35.355871 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.339754    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:35.356234 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.356178    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:35.489293 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.489250    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:35.686195 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.670982    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:59:35.932071    4577 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.772s"
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.932120    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"]
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.932332    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.932396    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.932449    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.932507    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:35.972056 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:35.932535    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"]
Oct 30 00:59:36.131504 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.131313    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:36.285145 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.275420    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:36.285145 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.275494    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:36.387016 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.386898    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" event={"ID":"9e69b637-61db-439b-a9d5-5878afeed344","Type":"ContainerStarted","Data":"5318f531d89f9dbe5cf9d3fc486b430d716ff5f6e32a5c1f0d5c5ce43d3662b6"}
Oct 30 00:59:36.484892 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.484823    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" event={"ID":"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4","Type":"ContainerStarted","Data":"1aca5dc4c05737f4110809695d6153fb8341ababf5efbc81c3579bf103e5ced4"}
Oct 30 00:59:36.600801 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.600750    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:36.750713 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.750668    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:36.968815 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.950343    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:36.970345 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:36.970301    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:37.159328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.146152    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 00:59:37.159328 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.146212    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 00:59:37.218873 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.218062    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"]
Oct 30 00:59:37.223492 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.223464    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"]
Oct 30 00:59:37.223811 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.223791    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.302575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.283072    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:59:37.302575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.284089    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:59:37.302575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.296351    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.302575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.296405    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5dc\" (UniqueName: \"kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.302575 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.296536    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.402886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.402636    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.402886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.402700    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5dc\" (UniqueName: \"kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.405178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.403171    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.405178 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.404331    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.664521 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.664459    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:37.745820 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.745793    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:37.765496 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.764667    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.765496 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.765442    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5dc\" (UniqueName: \"kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc\") pod \"maas-api-revoke-sub-78104af2-4cb7t\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:37.777529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.776615    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" event={"ID":"9e69b637-61db-439b-a9d5-5878afeed344","Type":"ContainerStarted","Data":"ac7c318df9fa778e688a7858600e5f18d1fdc554ad9a0a7af77b801f53ccee24"}
Oct 30 00:59:37.777529 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.777477    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:37.780686 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.780661    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" event={"ID":"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4","Type":"ContainerStarted","Data":"4e34bd8bb7fdb7026c60a2b35a9a226bd84dbb5e9244a0a4f30b4fbd28114f84"}
Oct 30 00:59:37.783586 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.783416    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:37.784092 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.784071    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:37.784618 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.784599    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:37.785714 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.785687    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 00:59:37.913972 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.913835    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" podStartSLOduration=7.913813915 podStartE2EDuration="7.913813915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:37.91315634 +0000 UTC m=+1085.942466572" watchObservedRunningTime="2026-10-30 00:59:37.913813915 +0000 UTC m=+1085.943124147"
Oct 30 00:59:37.959393 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:37.959319    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" podStartSLOduration=7.959303049 podStartE2EDuration="7.959303049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:37.958740827 +0000 UTC m=+1085.988051059" watchObservedRunningTime="2026-10-30 00:59:37.959303049 +0000 UTC m=+1085.988613281"
Oct 30 00:59:38.039033 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:38.038994    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:38.302909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:38.302860    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:38.505526 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:38.505480    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49ae627_e9ff_4695_b47b_e043b89ff169.slice/crio-db48f93741ffa7cd8d392f75e5e08439e37db7d0f410fe889c8aeb3c08c0af59 WatchSource:0}: Error finding container db48f93741ffa7cd8d392f75e5e08439e37db7d0f410fe889c8aeb3c08c0af59: Status 404 returned error can't find the container with id db48f93741ffa7cd8d392f75e5e08439e37db7d0f410fe889c8aeb3c08c0af59
Oct 30 00:59:38.507181 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:38.507157    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"]
Oct 30 00:59:38.797194 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:38.797095    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t" event={"ID":"a49ae627-e9ff-4695-b47b-e043b89ff169","Type":"ContainerStarted","Data":"db48f93741ffa7cd8d392f75e5e08439e37db7d0f410fe889c8aeb3c08c0af59"}
Oct 30 00:59:39.061927 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.060056    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:39.101869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.101489    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:39.101869 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.101576    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:39.727185 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.727155    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:39.806831 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.806789    4577 generic.go:356] "Generic (PLEG): container finished" podID="a49ae627-e9ff-4695-b47b-e043b89ff169" containerID="a3d3f75cce93e486f54c0cee64b819443fc16fa596793801eda14e6dbb16efc8" exitCode=0
Oct 30 00:59:39.807082 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.806895    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t" event={"ID":"a49ae627-e9ff-4695-b47b-e043b89ff169","Type":"ContainerDied","Data":"a3d3f75cce93e486f54c0cee64b819443fc16fa596793801eda14e6dbb16efc8"}
Oct 30 00:59:39.808524 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:39.808502    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 00:59:40.017126 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:40.017095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:40.974758 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:40.974731    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:41.077662 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.077566    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"]
Oct 30 00:59:41.078556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.078534    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a49ae627-e9ff-4695-b47b-e043b89ff169" containerName="revoke-keys"
Oct 30 00:59:41.078670 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.078558    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a49ae627-e9ff-4695-b47b-e043b89ff169" containerName="revoke-keys"
Oct 30 00:59:41.078719 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.078699    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a49ae627-e9ff-4695-b47b-e043b89ff169" containerName="revoke-keys"
Oct 30 00:59:41.082959 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.082940    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.102122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.102098    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"]
Oct 30 00:59:41.132062 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.132030    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca\") pod \"a49ae627-e9ff-4695-b47b-e043b89ff169\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") "
Oct 30 00:59:41.132258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.132182    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca\") pod \"a49ae627-e9ff-4695-b47b-e043b89ff169\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") "
Oct 30 00:59:41.132258 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.132237    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc\" (UniqueName: \"kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc\") pod \"a49ae627-e9ff-4695-b47b-e043b89ff169\" (UID: \"a49ae627-e9ff-4695-b47b-e043b89ff169\") "
Oct 30 00:59:41.132488 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.132458    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca" pod "a49ae627-e9ff-4695-b47b-e043b89ff169" (UID: "a49ae627-e9ff-4695-b47b-e043b89ff169"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:41.132960 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.132831    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:41.134658 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.134633    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca" pod "a49ae627-e9ff-4695-b47b-e043b89ff169" (UID: "a49ae627-e9ff-4695-b47b-e043b89ff169"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:41.135036 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.135018    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc" pod "a49ae627-e9ff-4695-b47b-e043b89ff169" (UID: "a49ae627-e9ff-4695-b47b-e043b89ff169"). InnerVolumeSpecName "kube-api-access-hj5dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:41.234035 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.233989    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234268 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234073    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234268 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234097    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqcc\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234268 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234116    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234372 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234279    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234412 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234395    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234442 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234417    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234478 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234442    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234541 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234529    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.234636 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234617    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a49ae627-e9ff-4695-b47b-e043b89ff169-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:41.234668 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.234642    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj5dc\" (UniqueName: \"kubernetes.io/projected/a49ae627-e9ff-4695-b47b-e043b89ff169-kube-api-access-hj5dc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:41.335568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335488    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.335568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335545    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.335568 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335566    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqcc\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.335830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335585    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.335830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335660    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.335830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335721    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.335830 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.335748    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336088 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336055    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336150 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336129    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336205 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336164    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336257 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336203    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336283    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336379 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336354    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.336735 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.336715    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.338128 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.338110    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.338275 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.338257    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.360834 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.360798    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.361024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.360968    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqcc\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.399584 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.399547    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:41.735883 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:41.735830    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926056e3_02b6_492f_8a1a_e93d5402b654.slice/crio-db987216a33e70b4a26f18ccad51ed384e142e268193b8797f9e68cd88e6a492 WatchSource:0}: Error finding container db987216a33e70b4a26f18ccad51ed384e142e268193b8797f9e68cd88e6a492: Status 404 returned error can't find the container with id db987216a33e70b4a26f18ccad51ed384e142e268193b8797f9e68cd88e6a492
Oct 30 00:59:41.737064 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.737040    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"]
Oct 30 00:59:41.738434 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.738406    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:41.738507 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.738464    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:41.738507 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.738492    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:41.825987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.825951    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t" event={"ID":"a49ae627-e9ff-4695-b47b-e043b89ff169","Type":"ContainerDied","Data":"db48f93741ffa7cd8d392f75e5e08439e37db7d0f410fe889c8aeb3c08c0af59"}
Oct 30 00:59:41.825987 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.826013    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db48f93741ffa7cd8d392f75e5e08439e37db7d0f410fe889c8aeb3c08c0af59"
Oct 30 00:59:41.826311 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.826089    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"
Oct 30 00:59:41.842739 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.838301    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" event={"ID":"926056e3-02b6-492f-8a1a-e93d5402b654","Type":"ContainerStarted","Data":"db987216a33e70b4a26f18ccad51ed384e142e268193b8797f9e68cd88e6a492"}
Oct 30 00:59:41.862517 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.862489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:41.991925 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:41.981816    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:42.875870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:42.874835    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" event={"ID":"926056e3-02b6-492f-8a1a-e93d5402b654","Type":"ContainerStarted","Data":"996675fcd1196c9e0d2e7dc9ab764743b16887757cde8524797108d43fcdb62f"}
Oct 30 00:59:42.974872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:42.971485    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podStartSLOduration=1.9714638660000001 podStartE2EDuration="1.971463866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:42.944569863 +0000 UTC m=+1090.973880106" watchObservedRunningTime="2026-10-30 00:59:42.971463866 +0000 UTC m=+1091.000774121"
Oct 30 00:59:43.408879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:43.401583    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:43.408879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:43.401720    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:43.408879 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:43.401781    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:43.927452 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:43.927420    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:44.403872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:44.400343    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:44.403872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:44.400441    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:45.406044 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:45.406008    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:45.407072 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:45.407041    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:46.044872 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.043976    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:46.404047 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.403948    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:46.404332 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.404285    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:46.650006 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.649952    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:46.673059 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.672948    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:46.673910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.673489    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:46.826990 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.826943    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:46.977252 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:46.977155    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:47.056824 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.056773    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:47.097515 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.065392    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:47.500832 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.500796    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:47.586880 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.548724    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:47.659242 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.657939    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:47.699909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.659869    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:59:47.699909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.673681    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:47.699909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.675575    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:47.699909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.678234    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:47.699909 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:47.687037    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:59:48.072895 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.059649    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:48.354013 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.353811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:48.380965 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.362276    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:48.380965 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.362338    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:48.419807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.411691    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:48.419807 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.411758    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:48.707343 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.707225    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 00:59:48.759438 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.736020    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 00:59:48.759438 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.752523    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 00:59:48.958928 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:48.957570    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 00:59:49.059910 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.059863    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 00:59:49.202894 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.196600    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:49.347933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.341305    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:49.366556 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.366523    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 00:59:49.629307 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.629229    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 00:59:49.635519 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.635494    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 00:59:49.638002 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.631762    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:49.652983 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.652939    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:49.824936 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.824024    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:49.994951 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:49.994919    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 00:59:50.238720 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:50.238684    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:50.420886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:50.379182    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 00:59:50.420886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:50.383619    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:50.420886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:50.402260    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:50.420886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:50.402309    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:50.420886 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:50.418337    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:51.127888 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:51.126812    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:51.399977 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:51.399879    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:51.399977 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:51.399937    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:52.046949 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.046911    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"]
Oct 30 00:59:52.215657 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.214153    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.294860    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.294978    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295019    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295070    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295101    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295151    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295210    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295384    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrlln\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.298931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.295424    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.317875 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.314644    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"]
Oct 30 00:59:52.396867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.396687    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.396867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.396755    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.397130 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.396921    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrlln\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.397130 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.396963    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.397130 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.397054    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.402867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.400926    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.402867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.400991    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.402867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.401047    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.402867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.401082    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.402867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.401970    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.403993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.404792    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.405145    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.405886    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.405934    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.406458    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.415865 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.413321    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.419867 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.416377    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.437781 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.437717    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.438701 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.438677    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrlln\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.576667 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.576574    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:52.930387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.930301    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:52.941329 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.941296    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"]
Oct 30 00:59:52.942049 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:52.942018    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b4a993_158d_453e_8818_aad628b254e9.slice/crio-881b889d591c438ae18830d60bb93505fe409b9f6a9fe3b539b5e5b2a8099b4f WatchSource:0}: Error finding container 881b889d591c438ae18830d60bb93505fe409b9f6a9fe3b539b5e5b2a8099b4f: Status 404 returned error can't find the container with id 881b889d591c438ae18830d60bb93505fe409b9f6a9fe3b539b5e5b2a8099b4f
Oct 30 00:59:52.945517 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.945475    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:52.945698 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.945672    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:52.945784 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:52.945730    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 00:59:53.165414 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.165381    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" event={"ID":"f7b4a993-158d-453e-8818-aad628b254e9","Type":"ContainerStarted","Data":"40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1"}
Oct 30 00:59:53.165590 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.165434    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" event={"ID":"f7b4a993-158d-453e-8818-aad628b254e9","Type":"ContainerStarted","Data":"881b889d591c438ae18830d60bb93505fe409b9f6a9fe3b539b5e5b2a8099b4f"}
Oct 30 00:59:53.232080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.232018    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podStartSLOduration=2.23199521 podStartE2EDuration="2.23199521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:53.227313949 +0000 UTC m=+1101.256624186" watchObservedRunningTime="2026-10-30 00:59:53.23199521 +0000 UTC m=+1101.261305446"
Oct 30 00:59:53.406600 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.406577    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:53.407109 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.407074    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:53.407972 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.407953    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 00:59:53.579024 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.578928    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 00:59:53.785409 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.785358    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"]
Oct 30 00:59:53.791512 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.791069    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.799360 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.798807    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"]
Oct 30 00:59:53.808084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.807945    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 00:59:53.808084 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.808071    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 00:59:53.835334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.835241    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nll4h\" (UniqueName: \"kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.835334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.835290    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.835535 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.835395    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.936589 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.936544    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.936788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.936715    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nll4h\" (UniqueName: \"kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.936788 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.936738    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.937274 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.937252    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.939334 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.939309    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:53.984877 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:53.975802    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nll4h\" (UniqueName: \"kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h\") pod \"maas-api-revoke-sub-87287c79-stlv4\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:54.116939 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:54.113387    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:54.231170 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:54.230287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:54.377585 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:54.377491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:54.481203 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:54.481169    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"]
Oct 30 00:59:54.481569 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:54.481541    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355e2206_9f05_456e_8d61_7b079a6b809c.slice/crio-ceaa4f3f9bd583429b4e971cb9d6e6120379020fcfda3082b9c8bc460a2a2643 WatchSource:0}: Error finding container ceaa4f3f9bd583429b4e971cb9d6e6120379020fcfda3082b9c8bc460a2a2643: Status 404 returned error can't find the container with id ceaa4f3f9bd583429b4e971cb9d6e6120379020fcfda3082b9c8bc460a2a2643
Oct 30 00:59:54.579390 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:54.579355    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 00:59:54.579683 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:54.579652    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 00:59:55.178599 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.178502    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:55.210441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.209482    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4" event={"ID":"355e2206-9f05-456e-8d61-7b079a6b809c","Type":"ContainerStarted","Data":"ef2afc0474dd4d7827bd417e5e6fc904c79ef5baba446c7833835f7d1ef62486"}
Oct 30 00:59:55.210441 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.209544    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4" event={"ID":"355e2206-9f05-456e-8d61-7b079a6b809c","Type":"ContainerStarted","Data":"ceaa4f3f9bd583429b4e971cb9d6e6120379020fcfda3082b9c8bc460a2a2643"}
Oct 30 00:59:55.577835 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.577795    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 00:59:55.578255 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.577884    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 00:59:55.612261 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.612226    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:55.972420 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:55.972336    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:56.078465 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.078410    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4" podStartSLOduration=3.07839439 podStartE2EDuration="3.07839439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:55.282470332 +0000 UTC m=+1103.311780565" watchObservedRunningTime="2026-10-30 00:59:56.07839439 +0000 UTC m=+1104.107704621"
Oct 30 00:59:56.083388 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.083363    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"]
Oct 30 00:59:56.098576 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.098548    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.105532 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.105490    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:56.117120 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.117094    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-52b947\""
Oct 30 00:59:56.126972 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.124996    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"]
Oct 30 00:59:56.142273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.142242    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"]
Oct 30 00:59:56.163368 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.163342    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.182924 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.182894    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-52b947-dockercfg-mpq8h\""
Oct 30 00:59:56.197080 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.197047    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"]
Oct 30 00:59:56.248870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.241030    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmwl9\" (UniqueName: \"kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9\") pod \"payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") " pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.248870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.241154    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.248870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.241196    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr25x\" (UniqueName: \"kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x\") pod \"payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.248870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.241536    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume\") pod \"payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") " pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.369050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.357872    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume\") pod \"payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") " pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.369050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.358122    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwl9\" (UniqueName: \"kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9\") pod \"payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") " pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.369050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.358186    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.369050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.358225    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr25x\" (UniqueName: \"kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x\") pod \"payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.369050 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.359594    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume\") pod \"payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") " pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.453919 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.453810    4577 generic.go:356] "Generic (PLEG): container finished" podID="355e2206-9f05-456e-8d61-7b079a6b809c" containerID="ef2afc0474dd4d7827bd417e5e6fc904c79ef5baba446c7833835f7d1ef62486" exitCode=0
Oct 30 00:59:56.454174 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.454150    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4" event={"ID":"355e2206-9f05-456e-8d61-7b079a6b809c","Type":"ContainerDied","Data":"ef2afc0474dd4d7827bd417e5e6fc904c79ef5baba446c7833835f7d1ef62486"}
Oct 30 00:59:56.456821 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.456794    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.487819 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.487757    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr25x\" (UniqueName: \"kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x\") pod \"payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.496888 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.489505    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmwl9\" (UniqueName: \"kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9\") pod \"payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") " pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.586922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.585981    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 00:59:56.586922 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.586040    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 00:59:56.635276 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.635233    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:56.719420 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.719384    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:56.983615 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:56.983570    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81da315_c2a3_425d_ad89_32d89283d3e3.slice/crio-c749022a840a252e8f52a93b90e50e7f6032d24ed9d3b2588aacda0680098779 WatchSource:0}: Error finding container c749022a840a252e8f52a93b90e50e7f6032d24ed9d3b2588aacda0680098779: Status 404 returned error can't find the container with id c749022a840a252e8f52a93b90e50e7f6032d24ed9d3b2588aacda0680098779
Oct 30 00:59:56.985273 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:56.985134    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"]
Oct 30 00:59:57.016835 ip-10-0-141-17 kubenswrapper[4577]: E1030 00:59:57.016801    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 00:59:57.091814 ip-10-0-141-17 kubenswrapper[4577]: W1030 00:59:57.091788    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd93e79_08f8_4c30_b0f0_120d370a93aa.slice/crio-71b51d6c5751c89f932ea047c1eb6ff08f60833447e7b3ba7a01253e82de4d35 WatchSource:0}: Error finding container 71b51d6c5751c89f932ea047c1eb6ff08f60833447e7b3ba7a01253e82de4d35: Status 404 returned error can't find the container with id 71b51d6c5751c89f932ea047c1eb6ff08f60833447e7b3ba7a01253e82de4d35
Oct 30 00:59:57.092953 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.092929    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"]
Oct 30 00:59:57.226018 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.225985    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:57.479630 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.479598    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv" event={"ID":"f81da315-c2a3-425d-ad89-32d89283d3e3","Type":"ContainerStarted","Data":"22bee88bbb98fcf57ab11af47701d38ee16e17cd86440d56d1f4a998c551928e"}
Oct 30 00:59:57.479820 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.479654    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv" event={"ID":"f81da315-c2a3-425d-ad89-32d89283d3e3","Type":"ContainerStarted","Data":"c749022a840a252e8f52a93b90e50e7f6032d24ed9d3b2588aacda0680098779"}
Oct 30 00:59:57.479901 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.479828    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 00:59:57.481619 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.481595    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj" event={"ID":"1dd93e79-08f8-4c30-b0f0-120d370a93aa","Type":"ContainerStarted","Data":"55b93fa0291a262f4f06991125b51642697d1469db11fdd98fb953a6b5e1545e"}
Oct 30 00:59:57.481756 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.481636    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj" event={"ID":"1dd93e79-08f8-4c30-b0f0-120d370a93aa","Type":"ContainerStarted","Data":"71b51d6c5751c89f932ea047c1eb6ff08f60833447e7b3ba7a01253e82de4d35"}
Oct 30 00:59:57.481814 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.481758    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 00:59:57.525462 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.525400    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv" podStartSLOduration=1.525379968 podStartE2EDuration="1.525379968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:57.524172688 +0000 UTC m=+1105.553482920" watchObservedRunningTime="2026-10-30 00:59:57.525379968 +0000 UTC m=+1105.554690201"
Oct 30 00:59:57.584321 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.584282    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 00:59:57.584494 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.584342    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 00:59:57.585382 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.585317    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj" podStartSLOduration=1.585296782 podStartE2EDuration="1.585296782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:57.568272995 +0000 UTC m=+1105.597583237" watchObservedRunningTime="2026-10-30 00:59:57.585296782 +0000 UTC m=+1105.614607015"
Oct 30 00:59:57.825122 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.825074    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:57.919606 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.919575    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca\") pod \"355e2206-9f05-456e-8d61-7b079a6b809c\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") "
Oct 30 00:59:57.919773 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.919718    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h\" (UniqueName: \"kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h\") pod \"355e2206-9f05-456e-8d61-7b079a6b809c\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") "
Oct 30 00:59:57.920039 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.920009    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca" pod "355e2206-9f05-456e-8d61-7b079a6b809c" (UID: "355e2206-9f05-456e-8d61-7b079a6b809c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:57.920185 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.920168    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca\") pod \"355e2206-9f05-456e-8d61-7b079a6b809c\" (UID: \"355e2206-9f05-456e-8d61-7b079a6b809c\") "
Oct 30 00:59:57.921933 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.921486    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:57.925994 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.925834    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca" pod "355e2206-9f05-456e-8d61-7b079a6b809c" (UID: "355e2206-9f05-456e-8d61-7b079a6b809c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:57.928031 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:57.927991    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h" pod "355e2206-9f05-456e-8d61-7b079a6b809c" (UID: "355e2206-9f05-456e-8d61-7b079a6b809c"). InnerVolumeSpecName "kube-api-access-nll4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:58.022785 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.022705    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nll4h\" (UniqueName: \"kubernetes.io/projected/355e2206-9f05-456e-8d61-7b079a6b809c-kube-api-access-nll4h\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:58.022785 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.022735    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/355e2206-9f05-456e-8d61-7b079a6b809c-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 00:59:58.490914 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.490875    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4" event={"ID":"355e2206-9f05-456e-8d61-7b079a6b809c","Type":"ContainerDied","Data":"ceaa4f3f9bd583429b4e971cb9d6e6120379020fcfda3082b9c8bc460a2a2643"}
Oct 30 00:59:58.491094 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.490936    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceaa4f3f9bd583429b4e971cb9d6e6120379020fcfda3082b9c8bc460a2a2643"
Oct 30 00:59:58.491094 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.490889    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"
Oct 30 00:59:58.577387 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.577339    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 00:59:58.577596 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:58.577422    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 00:59:59.149870 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:59.149349    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 00:59:59.577711 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:59.577670    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 00:59:59.577931 ip-10-0-141-17 kubenswrapper[4577]: I1030 00:59:59.577740    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 01:00:00.157836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.157798    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"]
Oct 30 01:00:00.159041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.159013    4577 cpu_manager.go:414] "containerMap: removing container" podUID="355e2206-9f05-456e-8d61-7b079a6b809c" containerName="revoke-keys"
Oct 30 01:00:00.159041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.159038    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="355e2206-9f05-456e-8d61-7b079a6b809c" containerName="revoke-keys"
Oct 30 01:00:00.159254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.159214    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="355e2206-9f05-456e-8d61-7b079a6b809c" containerName="revoke-keys"
Oct 30 01:00:00.165026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.165000    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.173032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.172667    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"]
Oct 30 01:00:00.202762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.202733    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:00:00.203999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.203969    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"]
Oct 30 01:00:00.206717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.206681    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.216216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.216186    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"]
Oct 30 01:00:00.216373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.216355    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.226558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.225997    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"]
Oct 30 01:00:00.232315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.232291    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"]
Oct 30 01:00:00.237202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.237178    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"]
Oct 30 01:00:00.242620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.242597    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.246744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.246724    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"]
Oct 30 01:00:00.264493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264458    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gb27\" (UniqueName: \"kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27\") pod \"maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.264633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264579    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.264633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264610    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888700-7kcfc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.264755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264666    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz64w\" (UniqueName: \"kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w\") pod \"maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.264755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264746    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgcc\" (UniqueName: \"kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.264871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264803    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.264924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264868    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.264924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.264913    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqzq\" (UniqueName: \"kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq\") pod \"maas-api-key-cleanup-29888700-7kcfc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366390    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqzq\" (UniqueName: \"kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq\") pod \"maas-api-key-cleanup-29888700-7kcfc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366475    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gb27\" (UniqueName: \"kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27\") pod \"maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366579    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366602    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888700-7kcfc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366654    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz64w\" (UniqueName: \"kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w\") pod \"maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366733    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgcc\" (UniqueName: \"kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366793    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.367867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.367588    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.368423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.366862    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.370912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.368601    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888700-7kcfc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.370912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.370179    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.379922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.371255    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.399674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.399639    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgcc\" (UniqueName: \"kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.400250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.400015    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqzq\" (UniqueName: \"kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq\") pod \"maas-api-key-cleanup-29888700-7kcfc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.400377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.400276    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz64w\" (UniqueName: \"kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w\") pod \"maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.401929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.401876    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gb27\" (UniqueName: \"kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27\") pod \"maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.510402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.510304    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:00.533155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.532911    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:00.545759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.545527    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:00.562828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.562360    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:00.914122 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:00:00.914087    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc020b2d5_639b_445a_ac97_6d83de097677.slice/crio-7e2dfbff39d8d1abdc1138d58df96b118231d0ffb8925f72a8c4c97d6dcb18fd WatchSource:0}: Error finding container 7e2dfbff39d8d1abdc1138d58df96b118231d0ffb8925f72a8c4c97d6dcb18fd: Status 404 returned error can't find the container with id 7e2dfbff39d8d1abdc1138d58df96b118231d0ffb8925f72a8c4c97d6dcb18fd
Oct 30 01:00:00.916563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.916532    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"]
Oct 30 01:00:00.964346 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:00:00.964314    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc561730b_d9da_466f_ab2e_fc4df2f69dcc.slice/crio-6197d2b6ca50e101a039580b5883b809a82e1f252818613adb68ceac649ecb1c WatchSource:0}: Error finding container 6197d2b6ca50e101a039580b5883b809a82e1f252818613adb68ceac649ecb1c: Status 404 returned error can't find the container with id 6197d2b6ca50e101a039580b5883b809a82e1f252818613adb68ceac649ecb1c
Oct 30 01:00:00.967571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.966917    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"]
Oct 30 01:00:00.974655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.974624    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"]
Oct 30 01:00:00.986719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:00.986685    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"]
Oct 30 01:00:01.025029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.024993    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 01:00:01.025539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.025520    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 01:00:01.026667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.026651    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 01:00:01.180336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.180307    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:01.522804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.522766    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerStarted","Data":"2c22f8692ea7797ef7a291237caac66f69ac8265aab65929293969d27d06f7dc"}
Oct 30 01:00:01.523044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.522828    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerStarted","Data":"6197d2b6ca50e101a039580b5883b809a82e1f252818613adb68ceac649ecb1c"}
Oct 30 01:00:01.524582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.524556    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerStarted","Data":"1f050e768e23ceeee4da7985c185d5832680aaad53fd6779d3a22c4c678be4fd"}
Oct 30 01:00:01.524708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.524599    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerStarted","Data":"7c4c59249b244661100cdb3e3d4d631ab1f8c902df5e51bfe023c0bf2c002f0c"}
Oct 30 01:00:01.526223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.526201    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerStarted","Data":"bc3ab86c2ae122075c161d39c64530849be969fc6f1efce5ec1156149dd2c01f"}
Oct 30 01:00:01.526796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.526240    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerStarted","Data":"09d0bfc9de47ba38ec2313d41f5d7dabc74a15fc8dd060b85997fefc2f3467fc"}
Oct 30 01:00:01.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.527879    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerStarted","Data":"377204c08cc14563fca0b57a8c9609dab930c3b03325fa5b1e045e10149a0e4e"}
Oct 30 01:00:01.528022 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.527917    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerStarted","Data":"7e2dfbff39d8d1abdc1138d58df96b118231d0ffb8925f72a8c4c97d6dcb18fd"}
Oct 30 01:00:01.569894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.569818    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" podStartSLOduration=1.569804902 podStartE2EDuration="1.569804902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:01.566021815 +0000 UTC m=+1109.595332046" watchObservedRunningTime="2026-10-30 01:00:01.569804902 +0000 UTC m=+1109.599115133"
Oct 30 01:00:01.611910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.611858    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" podStartSLOduration=1.6118302020000002 podStartE2EDuration="1.611830202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:01.60857145 +0000 UTC m=+1109.637881681" watchObservedRunningTime="2026-10-30 01:00:01.611830202 +0000 UTC m=+1109.641140433"
Oct 30 01:00:01.656630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.656564    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" podStartSLOduration=1.656545524 podStartE2EDuration="1.656545524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:01.652339655 +0000 UTC m=+1109.681649886" watchObservedRunningTime="2026-10-30 01:00:01.656545524 +0000 UTC m=+1109.685855757"
Oct 30 01:00:01.697483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:01.697420    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" podStartSLOduration=1.697407417 podStartE2EDuration="1.697407417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:01.695801526 +0000 UTC m=+1109.725111757" watchObservedRunningTime="2026-10-30 01:00:01.697407417 +0000 UTC m=+1109.726717647"
Oct 30 01:00:02.538682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.538653    4577 generic.go:356] "Generic (PLEG): container finished" podID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerID="1f050e768e23ceeee4da7985c185d5832680aaad53fd6779d3a22c4c678be4fd" exitCode=60
Oct 30 01:00:02.539156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.538737    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerDied","Data":"1f050e768e23ceeee4da7985c185d5832680aaad53fd6779d3a22c4c678be4fd"}
Oct 30 01:00:02.539156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.539029    4577 scope.go:122] "RemoveContainer" containerID="1f050e768e23ceeee4da7985c185d5832680aaad53fd6779d3a22c4c678be4fd"
Oct 30 01:00:02.540701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.540557    4577 generic.go:356] "Generic (PLEG): container finished" podID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerID="bc3ab86c2ae122075c161d39c64530849be969fc6f1efce5ec1156149dd2c01f" exitCode=60
Oct 30 01:00:02.540701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.540625    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerDied","Data":"bc3ab86c2ae122075c161d39c64530849be969fc6f1efce5ec1156149dd2c01f"}
Oct 30 01:00:02.540929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.540915    4577 scope.go:122] "RemoveContainer" containerID="bc3ab86c2ae122075c161d39c64530849be969fc6f1efce5ec1156149dd2c01f"
Oct 30 01:00:02.542458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.542436    4577 generic.go:356] "Generic (PLEG): container finished" podID="c020b2d5-639b-445a-ac97-6d83de097677" containerID="377204c08cc14563fca0b57a8c9609dab930c3b03325fa5b1e045e10149a0e4e" exitCode=60
Oct 30 01:00:02.542569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.542469    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerDied","Data":"377204c08cc14563fca0b57a8c9609dab930c3b03325fa5b1e045e10149a0e4e"}
Oct 30 01:00:02.542731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.542697    4577 scope.go:122] "RemoveContainer" containerID="377204c08cc14563fca0b57a8c9609dab930c3b03325fa5b1e045e10149a0e4e"
Oct 30 01:00:02.544421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.544402    4577 generic.go:356] "Generic (PLEG): container finished" podID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerID="2c22f8692ea7797ef7a291237caac66f69ac8265aab65929293969d27d06f7dc" exitCode=60
Oct 30 01:00:02.544490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.544472    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerDied","Data":"2c22f8692ea7797ef7a291237caac66f69ac8265aab65929293969d27d06f7dc"}
Oct 30 01:00:02.544673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.544657    4577 scope.go:122] "RemoveContainer" containerID="2c22f8692ea7797ef7a291237caac66f69ac8265aab65929293969d27d06f7dc"
Oct 30 01:00:02.771622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.771594    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:02.978644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:02.978615    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:03.577030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.576992    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerStarted","Data":"5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570"}
Oct 30 01:00:03.580699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.580327    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerStarted","Data":"3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"}
Oct 30 01:00:03.580699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.580596    4577 scope.go:122] "RemoveContainer" containerID="3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"
Oct 30 01:00:03.581013 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:03.580985    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75_odh-ai-gateway-infra(c020b2d5-639b-445a-ac97-6d83de097677)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" podUID="c020b2d5-639b-445a-ac97-6d83de097677"
Oct 30 01:00:03.583423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.583397    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerStarted","Data":"24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126"}
Oct 30 01:00:03.586249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.586230    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerStarted","Data":"f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"}
Oct 30 01:00:03.586536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.586523    4577 scope.go:122] "RemoveContainer" containerID="f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"
Oct 30 01:00:03.586784 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:03.586768    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22_odh-ai-gateway-infra(83d9397c-c9ad-448d-bd71-e4a05e339e9c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c"
Oct 30 01:00:03.632158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:03.632119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:04.447679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.447637    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:04.652870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.652654    4577 generic.go:356] "Generic (PLEG): container finished" podID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerID="f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc" exitCode=60
Oct 30 01:00:04.652870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.652818    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerDied","Data":"f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"}
Oct 30 01:00:04.653675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.652901    4577 scope.go:122] "RemoveContainer" containerID="1f050e768e23ceeee4da7985c185d5832680aaad53fd6779d3a22c4c678be4fd"
Oct 30 01:00:04.653675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.653485    4577 scope.go:122] "RemoveContainer" containerID="f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"
Oct 30 01:00:04.658872 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:04.653965    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22_odh-ai-gateway-infra(83d9397c-c9ad-448d-bd71-e4a05e339e9c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c"
Oct 30 01:00:04.714476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.714392    4577 generic.go:356] "Generic (PLEG): container finished" podID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerID="5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570" exitCode=60
Oct 30 01:00:04.736866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.721782    4577 scope.go:122] "RemoveContainer" containerID="5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570"
Oct 30 01:00:04.736866 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:04.722244    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2_odh-ai-gateway-infra(a5c03ff3-80fe-445f-a0ee-98adfb766db6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6"
Oct 30 01:00:04.736866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.723568    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerDied","Data":"5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570"}
Oct 30 01:00:04.751882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.747109    4577 scope.go:122] "RemoveContainer" containerID="bc3ab86c2ae122075c161d39c64530849be969fc6f1efce5ec1156149dd2c01f"
Oct 30 01:00:04.768103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.767327    4577 generic.go:356] "Generic (PLEG): container finished" podID="c020b2d5-639b-445a-ac97-6d83de097677" containerID="3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa" exitCode=60
Oct 30 01:00:04.768103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.767481    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerDied","Data":"3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"}
Oct 30 01:00:04.768103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.767892    4577 scope.go:122] "RemoveContainer" containerID="3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"
Oct 30 01:00:04.769232 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:04.768971    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75_odh-ai-gateway-infra(c020b2d5-639b-445a-ac97-6d83de097677)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" podUID="c020b2d5-639b-445a-ac97-6d83de097677"
Oct 30 01:00:04.775140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.775098    4577 generic.go:356] "Generic (PLEG): container finished" podID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerID="24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126" exitCode=60
Oct 30 01:00:04.775273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.775239    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerDied","Data":"24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126"}
Oct 30 01:00:04.775559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.775517    4577 scope.go:122] "RemoveContainer" containerID="24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126"
Oct 30 01:00:04.775973 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:04.775889    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888700-7kcfc_odh-ai-gateway-infra(c561730b-d9da-466f-ab2e-fc4df2f69dcc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc"
Oct 30 01:00:04.782658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.782635    4577 scope.go:122] "RemoveContainer" containerID="377204c08cc14563fca0b57a8c9609dab930c3b03325fa5b1e045e10149a0e4e"
Oct 30 01:00:04.797403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.797381    4577 scope.go:122] "RemoveContainer" containerID="2c22f8692ea7797ef7a291237caac66f69ac8265aab65929293969d27d06f7dc"
Oct 30 01:00:04.896241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:04.896209    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:05.838866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:05.838171    4577 scope.go:122] "RemoveContainer" containerID="5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570"
Oct 30 01:00:05.838866 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:05.838616    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2_odh-ai-gateway-infra(a5c03ff3-80fe-445f-a0ee-98adfb766db6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6"
Oct 30 01:00:05.868645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:05.868617    4577 scope.go:122] "RemoveContainer" containerID="3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"
Oct 30 01:00:05.886325 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:05.886286    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75_odh-ai-gateway-infra(c020b2d5-639b-445a-ac97-6d83de097677)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" podUID="c020b2d5-639b-445a-ac97-6d83de097677"
Oct 30 01:00:05.886992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:05.886881    4577 scope.go:122] "RemoveContainer" containerID="24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126"
Oct 30 01:00:05.887527 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:05.887503    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888700-7kcfc_odh-ai-gateway-infra(c561730b-d9da-466f-ab2e-fc4df2f69dcc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc"
Oct 30 01:00:05.900904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:05.890492    4577 scope.go:122] "RemoveContainer" containerID="f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"
Oct 30 01:00:05.900904 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:05.890792    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22_odh-ai-gateway-infra(83d9397c-c9ad-448d-bd71-e4a05e339e9c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c"
Oct 30 01:00:06.177927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:06.147601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:07.019868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.019573    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:07.204551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.204517    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"]
Oct 30 01:00:07.216556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.216526    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.250088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.250035    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:07.273083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.258383    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p8xs2\""
Oct 30 01:00:07.273318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.258465    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-52b947-serving-cert\""
Oct 30 01:00:07.273477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.258603    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 01:00:07.412877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.407730    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.444115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.425548    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsxlp\" (UniqueName: \"kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.472111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.472078    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.472421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.472399    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.493940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.493904    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"]
Oct 30 01:00:07.611930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.580169    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.612602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.612564    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.613444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.613418    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsxlp\" (UniqueName: \"kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.613719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.613697    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.614679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.614657    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.651995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.640051    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.652252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.651890    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:07.683884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.674614    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.712871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.712699    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsxlp\" (UniqueName: \"kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp\") pod \"maas-api-e2e-api-a-52b947-dbd447ddb-jsccn\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.783872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.783142    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:07.851072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:07.851024    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 01:00:08.381872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.376935    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:08.547873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.547437    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 01:00:08.572870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.567375    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 01:00:08.675919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.675287    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:08.966950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.856954    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:08.966950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.901314    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:08.966950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.901381    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:08.966950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:08.920419    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 01:00:09.170628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.143460    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:09.397909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.377952    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:09.401349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.401310    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:00:09.402484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.402456    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:09.440871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.438742    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"]
Oct 30 01:00:09.517871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.505580    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:09.517871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.505640    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:09.685931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.661988    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:09.685931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.677473    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:09.685931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.684996    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:00:09.885461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.857759    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 01:00:09.914422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.914332    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:09.948927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:09.922613    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:00:10.040873 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:00:10.036974    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c060341_33b8_418b_945d_e3af71509aa4.slice/crio-c9ac745d4017228bdb86fc8daaa61283e7bdef69f3f5cfe0f075e982a5a61a3d WatchSource:0}: Error finding container c9ac745d4017228bdb86fc8daaa61283e7bdef69f3f5cfe0f075e982a5a61a3d: Status 404 returned error can't find the container with id c9ac745d4017228bdb86fc8daaa61283e7bdef69f3f5cfe0f075e982a5a61a3d
Oct 30 01:00:10.490911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:10.482953    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:10.695124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:10.693949    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:10.874926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:10.870957    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:10.874926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:10.871022    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:10.874926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:10.872301    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:11.228913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:11.218264    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 01:00:11.237419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:11.237377    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn" event={"ID":"4c060341-33b8-418b-945d-e3af71509aa4","Type":"ContainerStarted","Data":"c9ac745d4017228bdb86fc8daaa61283e7bdef69f3f5cfe0f075e982a5a61a3d"}
Oct 30 01:00:11.239162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:11.239140    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:00:11.240647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:11.240621    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:00:11.587673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:11.587567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:13.240368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:13.240339    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:14.094259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.094224    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"]
Oct 30 01:00:14.103524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.103490    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"]
Oct 30 01:00:14.103722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.103624    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196197    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngmwm\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196250    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196271    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196356    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196393    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196453    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196534    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196597    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.196739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.196624    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.297745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.297705    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.297769    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.297814    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.297917    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.297981    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298016    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298064    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngmwm\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298092    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298115    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298260    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298395    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298562    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298624    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.298936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.298775    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.300582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.300560    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.301015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.300995    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.310409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.310382    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn" event={"ID":"4c060341-33b8-418b-945d-e3af71509aa4","Type":"ContainerStarted","Data":"5ab129f340236d2657e578a803a4f789e5996d266f56cfc68aa2dea64130fd75"}
Oct 30 01:00:14.310582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.310526    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:14.323422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.323389    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.323837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.323817    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngmwm\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm\") pod \"maas-default-gateway-openshift-default-fcb75c45f-lqpbn\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.381965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.381822    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn" podStartSLOduration=5.075581574 podStartE2EDuration="8.381800386s" totalImagesPullingTime="3.306218812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:14.366659492 +0000 UTC m=+1122.395969723" watchObservedRunningTime="2026-10-30 01:00:14.381800386 +0000 UTC m=+1122.411110620"
Oct 30 01:00:14.436428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.436397    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:14.491871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.485128    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:14.972799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.972771    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"]
Oct 30 01:00:14.973481 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:00:14.973454    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3c8e5c_7027_4961_b0bb_0705746ab4c8.slice/crio-a9a6f642fe5a83d0c8de108d65b1446ebf29ddea24108ba60ce1e66313705d18 WatchSource:0}: Error finding container a9a6f642fe5a83d0c8de108d65b1446ebf29ddea24108ba60ce1e66313705d18: Status 404 returned error can't find the container with id a9a6f642fe5a83d0c8de108d65b1446ebf29ddea24108ba60ce1e66313705d18
Oct 30 01:00:14.976697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.976657    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:14.976782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.976748    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:14.976826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:14.976794    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:15.003882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.003825    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:15.119866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.113504    4577 scope.go:122] "RemoveContainer" containerID="f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"
Oct 30 01:00:15.322593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.322559    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" event={"ID":"9f3c8e5c-7027-4961-b0bb-0705746ab4c8","Type":"ContainerStarted","Data":"bf5b1cbcd3e4ad4e9360ec4c14c56595db2e19aed7a412b417e6d91da416abf5"}
Oct 30 01:00:15.323085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.322613    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" event={"ID":"9f3c8e5c-7027-4961-b0bb-0705746ab4c8","Type":"ContainerStarted","Data":"a9a6f642fe5a83d0c8de108d65b1446ebf29ddea24108ba60ce1e66313705d18"}
Oct 30 01:00:15.327607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.327577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:15.374607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.374527    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podStartSLOduration=1.374506558 podStartE2EDuration="1.374506558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:15.372146772 +0000 UTC m=+1123.401456999" watchObservedRunningTime="2026-10-30 01:00:15.374506558 +0000 UTC m=+1123.403816793"
Oct 30 01:00:15.436533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:15.436502    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:16.104613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.104583    4577 scope.go:122] "RemoveContainer" containerID="5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570"
Oct 30 01:00:16.104764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.104721    4577 scope.go:122] "RemoveContainer" containerID="24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126"
Oct 30 01:00:16.104883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.104835    4577 scope.go:122] "RemoveContainer" containerID="3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"
Oct 30 01:00:16.333408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.333373    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerStarted","Data":"e74d514bfa2610ecd206d87acfe6a5b572f61fff1ae6006f5a02866616df5cb1"}
Oct 30 01:00:16.438968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.438940    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:16.439134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.438991    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:16.784901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:16.784803    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:17.369228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.368793    4577 generic.go:356] "Generic (PLEG): container finished" podID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerID="e74d514bfa2610ecd206d87acfe6a5b572f61fff1ae6006f5a02866616df5cb1" exitCode=60
Oct 30 01:00:17.369228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.368952    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerDied","Data":"e74d514bfa2610ecd206d87acfe6a5b572f61fff1ae6006f5a02866616df5cb1"}
Oct 30 01:00:17.369228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.369048    4577 scope.go:122] "RemoveContainer" containerID="f3e725377e90e8a0d7a8e8136ce661b33f8f07ed727046d9fb160f346805c5dc"
Oct 30 01:00:17.369835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.369523    4577 scope.go:122] "RemoveContainer" containerID="e74d514bfa2610ecd206d87acfe6a5b572f61fff1ae6006f5a02866616df5cb1"
Oct 30 01:00:17.369926 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:17.369870    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22_odh-ai-gateway-infra(83d9397c-c9ad-448d-bd71-e4a05e339e9c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c"
Oct 30 01:00:17.382297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.382262    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerStarted","Data":"04780772e9d5d9f5461bdc830983f29e884df3510ba07f16bc45055ae47a1c23"}
Oct 30 01:00:17.385804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.385771    4577 generic.go:356] "Generic (PLEG): container finished" podID="c020b2d5-639b-445a-ac97-6d83de097677" containerID="508e425151580594d0bdf426ee518b4f8a1cc9c564bd65d047909051bfc40a33" exitCode=60
Oct 30 01:00:17.385925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.385811    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerDied","Data":"508e425151580594d0bdf426ee518b4f8a1cc9c564bd65d047909051bfc40a33"}
Oct 30 01:00:17.386457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.386434    4577 scope.go:122] "RemoveContainer" containerID="508e425151580594d0bdf426ee518b4f8a1cc9c564bd65d047909051bfc40a33"
Oct 30 01:00:17.386885 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:17.386859    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75_odh-ai-gateway-infra(c020b2d5-639b-445a-ac97-6d83de097677)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" podUID="c020b2d5-639b-445a-ac97-6d83de097677"
Oct 30 01:00:17.389494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.389453    4577 generic.go:356] "Generic (PLEG): container finished" podID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerID="29ae027a0888b0104ba1c4bf691c5e48f1f69670805b14014d701470a1a8ecaf" exitCode=60
Oct 30 01:00:17.389601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.389559    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerDied","Data":"29ae027a0888b0104ba1c4bf691c5e48f1f69670805b14014d701470a1a8ecaf"}
Oct 30 01:00:17.390145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.390124    4577 scope.go:122] "RemoveContainer" containerID="29ae027a0888b0104ba1c4bf691c5e48f1f69670805b14014d701470a1a8ecaf"
Oct 30 01:00:17.390478 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:17.390449    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29888700-7kcfc_odh-ai-gateway-infra(c561730b-d9da-466f-ab2e-fc4df2f69dcc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc"
Oct 30 01:00:17.409136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.409103    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"]
Oct 30 01:00:17.437410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.437377    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:17.437594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.437426    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:17.485129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.482627    4577 scope.go:122] "RemoveContainer" containerID="3588ec96feac51d1f8422f0d18bc0ba568023b9f747c1f0de9472958f6e185fa"
Oct 30 01:00:17.498631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:17.498607    4577 scope.go:122] "RemoveContainer" containerID="24f3f507f8ce4b12016fc49ce4955ea60693d7efc6ee4d82556c505d2b57e126"
Oct 30 01:00:18.408697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.408668    4577 generic.go:356] "Generic (PLEG): container finished" podID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerID="04780772e9d5d9f5461bdc830983f29e884df3510ba07f16bc45055ae47a1c23" exitCode=60
Oct 30 01:00:18.409125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.408758    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerDied","Data":"04780772e9d5d9f5461bdc830983f29e884df3510ba07f16bc45055ae47a1c23"}
Oct 30 01:00:18.409125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.408824    4577 scope.go:122] "RemoveContainer" containerID="5f2a91ea45fdfa6c8ec4ab9b958e45188fe4915fc0bbf90c2019071c93d80570"
Oct 30 01:00:18.409322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.409303    4577 scope.go:122] "RemoveContainer" containerID="04780772e9d5d9f5461bdc830983f29e884df3510ba07f16bc45055ae47a1c23"
Oct 30 01:00:18.409651 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:18.409629    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2_odh-ai-gateway-infra(a5c03ff3-80fe-445f-a0ee-98adfb766db6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6"
Oct 30 01:00:18.438831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.436794    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:18.438831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.436887    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:18.496867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.496383    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"]
Oct 30 01:00:18.510831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.510098    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:18.803285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.800499    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"]
Oct 30 01:00:18.827582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.826590    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"]
Oct 30 01:00:18.900913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.900884    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:18.989779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.989741    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w\" (UniqueName: \"kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w\") pod \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") "
Oct 30 01:00:18.989934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.989880    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca\") pod \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\" (UID: \"83d9397c-c9ad-448d-bd71-e4a05e339e9c\") "
Oct 30 01:00:18.991044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.990999    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca" pod "83d9397c-c9ad-448d-bd71-e4a05e339e9c" (UID: "83d9397c-c9ad-448d-bd71-e4a05e339e9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:00:18.999032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:18.998993    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w" pod "83d9397c-c9ad-448d-bd71-e4a05e339e9c" (UID: "83d9397c-c9ad-448d-bd71-e4a05e339e9c"). InnerVolumeSpecName "kube-api-access-pz64w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:00:19.030871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.030013    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:19.091917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.091701    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq\" (UniqueName: \"kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq\") pod \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") "
Oct 30 01:00:19.091917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.091893    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca\") pod \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\" (UID: \"c561730b-d9da-466f-ab2e-fc4df2f69dcc\") "
Oct 30 01:00:19.101900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.092696    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83d9397c-c9ad-448d-bd71-e4a05e339e9c-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.101900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.092721    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz64w\" (UniqueName: \"kubernetes.io/projected/83d9397c-c9ad-448d-bd71-e4a05e339e9c-kube-api-access-pz64w\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.101900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.093085    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca" pod "c561730b-d9da-466f-ab2e-fc4df2f69dcc" (UID: "c561730b-d9da-466f-ab2e-fc4df2f69dcc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:00:19.105166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.105033    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq" pod "c561730b-d9da-466f-ab2e-fc4df2f69dcc" (UID: "c561730b-d9da-466f-ab2e-fc4df2f69dcc"). InnerVolumeSpecName "kube-api-access-xkqzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:00:19.195648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.195600    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkqzq\" (UniqueName: \"kubernetes.io/projected/c561730b-d9da-466f-ab2e-fc4df2f69dcc-kube-api-access-xkqzq\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.195648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.195625    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c561730b-d9da-466f-ab2e-fc4df2f69dcc-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.201409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.201065    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:19.297575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.296522    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc\" (UniqueName: \"kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc\") pod \"c020b2d5-639b-445a-ac97-6d83de097677\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") "
Oct 30 01:00:19.297575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.296669    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca\") pod \"c020b2d5-639b-445a-ac97-6d83de097677\" (UID: \"c020b2d5-639b-445a-ac97-6d83de097677\") "
Oct 30 01:00:19.297980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.297953    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca" pod "c020b2d5-639b-445a-ac97-6d83de097677" (UID: "c020b2d5-639b-445a-ac97-6d83de097677"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:00:19.300720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.300691    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc" pod "c020b2d5-639b-445a-ac97-6d83de097677" (UID: "c020b2d5-639b-445a-ac97-6d83de097677"). InnerVolumeSpecName "kube-api-access-9jgcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:00:19.399451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.399365    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c020b2d5-639b-445a-ac97-6d83de097677-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.399451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.399408    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jgcc\" (UniqueName: \"kubernetes.io/projected/c020b2d5-639b-445a-ac97-6d83de097677-kube-api-access-9jgcc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.423858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.423825    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"
Oct 30 01:00:19.424339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.423873    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22" event={"ID":"83d9397c-c9ad-448d-bd71-e4a05e339e9c","Type":"ContainerDied","Data":"7c4c59249b244661100cdb3e3d4d631ab1f8c902df5e51bfe023c0bf2c002f0c"}
Oct 30 01:00:19.424339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.423951    4577 scope.go:122] "RemoveContainer" containerID="e74d514bfa2610ecd206d87acfe6a5b572f61fff1ae6006f5a02866616df5cb1"
Oct 30 01:00:19.431130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.431099    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"
Oct 30 01:00:19.431242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.431121    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75" event={"ID":"c020b2d5-639b-445a-ac97-6d83de097677","Type":"ContainerDied","Data":"7e2dfbff39d8d1abdc1138d58df96b118231d0ffb8925f72a8c4c97d6dcb18fd"}
Oct 30 01:00:19.432978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.432956    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"
Oct 30 01:00:19.433261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.433237    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc" event={"ID":"c561730b-d9da-466f-ab2e-fc4df2f69dcc","Type":"ContainerDied","Data":"6197d2b6ca50e101a039580b5883b809a82e1f252818613adb68ceac649ecb1c"}
Oct 30 01:00:19.440551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.440139    4577 scope.go:122] "RemoveContainer" containerID="508e425151580594d0bdf426ee518b4f8a1cc9c564bd65d047909051bfc40a33"
Oct 30 01:00:19.440551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.440172    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:19.440551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.440236    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:19.485318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.485287    4577 scope.go:122] "RemoveContainer" containerID="29ae027a0888b0104ba1c4bf691c5e48f1f69670805b14014d701470a1a8ecaf"
Oct 30 01:00:19.507018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.506979    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"]
Oct 30 01:00:19.515324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.515301    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-9bfe36-29888700-zjp22"]
Oct 30 01:00:19.615205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.615174    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"]
Oct 30 01:00:19.624510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.624487    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:19.625726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.625707    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888700-7kcfc"]
Oct 30 01:00:19.684380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.684303    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"]
Oct 30 01:00:19.692124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.692094    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888700-mpr75"]
Oct 30 01:00:19.704368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.704343    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca\") pod \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") "
Oct 30 01:00:19.704492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.704468    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27\" (UniqueName: \"kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27\") pod \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\" (UID: \"a5c03ff3-80fe-445f-a0ee-98adfb766db6\") "
Oct 30 01:00:19.704870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.704824    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca" pod "a5c03ff3-80fe-445f-a0ee-98adfb766db6" (UID: "a5c03ff3-80fe-445f-a0ee-98adfb766db6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:00:19.706740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.706716    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27" pod "a5c03ff3-80fe-445f-a0ee-98adfb766db6" (UID: "a5c03ff3-80fe-445f-a0ee-98adfb766db6"). InnerVolumeSpecName "kube-api-access-8gb27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:00:19.806505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.806457    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gb27\" (UniqueName: \"kubernetes.io/projected/a5c03ff3-80fe-445f-a0ee-98adfb766db6-kube-api-access-8gb27\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:19.806505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:19.806484    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c03ff3-80fe-445f-a0ee-98adfb766db6-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:00:20.068864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.068437    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:20.116172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.116131    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" path="/var/lib/kubelet/pods/83d9397c-c9ad-448d-bd71-e4a05e339e9c/volumes"
Oct 30 01:00:20.116930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.116907    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c020b2d5-639b-445a-ac97-6d83de097677" path="/var/lib/kubelet/pods/c020b2d5-639b-445a-ac97-6d83de097677/volumes"
Oct 30 01:00:20.117680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.117658    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" path="/var/lib/kubelet/pods/c561730b-d9da-466f-ab2e-fc4df2f69dcc/volumes"
Oct 30 01:00:20.182678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.182642    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:20.329338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.329252    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:00:20.407992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.407900    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:20.436860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.436800    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:20.437315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.436900    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:20.485979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.485951    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"
Oct 30 01:00:20.487820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.485951    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2" event={"ID":"a5c03ff3-80fe-445f-a0ee-98adfb766db6","Type":"ContainerDied","Data":"09d0bfc9de47ba38ec2313d41f5d7dabc74a15fc8dd060b85997fefc2f3467fc"}
Oct 30 01:00:20.488048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.488030    4577 scope.go:122] "RemoveContainer" containerID="04780772e9d5d9f5461bdc830983f29e884df3510ba07f16bc45055ae47a1c23"
Oct 30 01:00:20.682118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.681990    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"]
Oct 30 01:00:20.690059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:20.689951    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-39b241-29888700-gm8d2"]
Oct 30 01:00:21.437766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.437693    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:21.437766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.437744    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:21.503200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.503166    4577 generic.go:356] "Generic (PLEG): container finished" podID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerID="65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1" exitCode=0
Oct 30 01:00:21.503648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.503246    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" event={"ID":"fe8e2ee4-1059-413c-90d7-feaf02e70ed3","Type":"ContainerDied","Data":"65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"}
Oct 30 01:00:21.503855 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.503821    4577 scope.go:122] "RemoveContainer" containerID="65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"
Oct 30 01:00:21.505511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.505482    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:21.505616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.505555    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:21.505616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:21.505592    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:22.113738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.113700    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" path="/var/lib/kubelet/pods/a5c03ff3-80fe-445f-a0ee-98adfb766db6/volumes"
Oct 30 01:00:22.192534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.192494    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:22.437729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.437652    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 01:00:22.437729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.437710    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 01:00:22.514353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.514318    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" event={"ID":"fe8e2ee4-1059-413c-90d7-feaf02e70ed3","Type":"ContainerStarted","Data":"e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb"}
Oct 30 01:00:22.638803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.638775    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 01:00:22.639876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.639819    4577 patch_prober.go:29] interesting pod/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 01:00:22.640023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:22.639894    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 01:00:23.638343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:23.638307    4577 patch_prober.go:29] interesting pod/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 01:00:23.638745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:23.638365    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 01:00:24.001870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:23.991298    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:24.438173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.438093    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:24.438173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.438157    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:24.493178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.493145    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:24.493764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.493742    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:24.494703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.494683    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:00:24.522939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.522914    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:24.651869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.647221    4577 patch_prober.go:29] interesting pod/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 01:00:24.651869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:24.647283    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 01:00:25.208504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:25.208469    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:25.638997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:25.638966    4577 patch_prober.go:29] interesting pod/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 01:00:25.639150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:25.639016    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 01:00:25.798277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:25.798232    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:26.215640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.215607    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"]
Oct 30 01:00:26.216298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216282    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.216349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216302    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.216349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216315    4577 cpu_manager.go:414] "containerMap: removing container" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.216349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216322    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.216349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216336    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.216349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216342    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216359    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216364    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216380    4577 cpu_manager.go:414] "containerMap: removing container" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216384    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216395    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216399    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216411    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216415    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216423    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216427    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216437    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.216500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216440    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216532    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216541    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216547    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216554    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216559    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216564    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216569    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216573    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.216853 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.216582    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.222974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.222952    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.245868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.245694    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"]
Oct 30 01:00:26.314892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.314807    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"]
Oct 30 01:00:26.315801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.315778    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.316894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.316874    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.317084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317057    4577 cpu_manager.go:414] "containerMap: removing container" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.317178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317167    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83d9397c-c9ad-448d-bd71-e4a05e339e9c" containerName="cleanup"
Oct 30 01:00:26.317243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.316787    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.317314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317295    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.317453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317438    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.317602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317572    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.317716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317704    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.317874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317862    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.317986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317959    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.318144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.317889    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45lwf\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.318233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.318213    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.318315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.318292    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.318370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.318333    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.318614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.318590    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c561730b-d9da-466f-ab2e-fc4df2f69dcc" containerName="cleanup"
Oct 30 01:00:26.318718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.318709    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c020b2d5-639b-445a-ac97-6d83de097677" containerName="cleanup"
Oct 30 01:00:26.318819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.318769    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5c03ff3-80fe-445f-a0ee-98adfb766db6" containerName="cleanup"
Oct 30 01:00:26.323588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.323574    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"]
Oct 30 01:00:26.323739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.323716    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420061    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420106    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.420310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420219    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.420310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420261    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.420310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420290    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420454    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420494    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420553    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.420685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420619    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.420740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420714    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.420823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420795    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420829    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.420991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.420958    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz7c6\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.421109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421039    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421116    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421202    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421281    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45lwf\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421309    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.421374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421354    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.421525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421511    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421536    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.421722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.421701    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.422833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.422805    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.423472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.423447    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.449867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.444983    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.485676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.485645    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45lwf\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.522680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522651    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.522680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522682    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.522917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522713    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz7c6\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.522917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522793    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.522917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522815    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.522917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522880    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.522973    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523088    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523111    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523112    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523198    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523189    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523475    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.523588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.523572    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.525543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.525518    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.525777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.525757    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.539298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.539279    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:26.580299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.580225    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.583205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.583070    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz7c6\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.638962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.638928    4577 patch_prober.go:29] interesting pod/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 01:00:26.639081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.638988    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 01:00:26.639788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.639585    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:26.912304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.912272    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"]
Oct 30 01:00:26.916044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.916005    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:26.916129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.916092    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:26.916169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:26.916133    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:27.018343 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:27.018229    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:00:27.100536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.100241    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"]
Oct 30 01:00:27.100902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.100867    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:27.101139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.101103    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:27.101309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.101279    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:27.327643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.326941    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 01:00:27.583303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.577017    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:27.603228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.603194    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" event={"ID":"f83157ed-e10d-4f90-a3bc-bec67e35b3b8","Type":"ContainerStarted","Data":"b7ce996bcf2441757a481e79c6735e4b6cf489fad26a9ffeef2fe1bb5dd1a90e"}
Oct 30 01:00:27.603430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.603254    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" event={"ID":"f83157ed-e10d-4f90-a3bc-bec67e35b3b8","Type":"ContainerStarted","Data":"7a77e4f10f854e585e95e42d1ab36903d671b42b5b1ae2bf358ad9f5f5b40850"}
Oct 30 01:00:27.609921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.608830    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" event={"ID":"71d6f8ed-df9e-4e61-951d-c4490161d7ae","Type":"ContainerStarted","Data":"b216588f397d60972b5384bec8afab494643e0e8b4b44e8f793c8a7913e5bd34"}
Oct 30 01:00:27.609921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.608936    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" event={"ID":"71d6f8ed-df9e-4e61-951d-c4490161d7ae","Type":"ContainerStarted","Data":"3d8d2e803b3b1ee50c59542fac8617acc8620ef90a59c9c3bfe225c22e8bf60d"}
Oct 30 01:00:27.638922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.638444    4577 patch_prober.go:29] interesting pod/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 01:00:27.638922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.638514    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 01:00:27.640499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.640476    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:27.676738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.676661    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podStartSLOduration=1.676638407 podStartE2EDuration="1.676638407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:27.655392767 +0000 UTC m=+1135.684702998" watchObservedRunningTime="2026-10-30 01:00:27.676638407 +0000 UTC m=+1135.705948640"
Oct 30 01:00:27.741140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:27.741040    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podStartSLOduration=1.7410173709999999 podStartE2EDuration="1.741017371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:27.708157229 +0000 UTC m=+1135.737467460" watchObservedRunningTime="2026-10-30 01:00:27.741017371 +0000 UTC m=+1135.770327603"
Oct 30 01:00:28.540402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.540367    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:28.542662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.542637    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:28.542810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.542694    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:28.641701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.641663    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:28.641919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.641711    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:28.643551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.643527    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 01:00:28.645036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:28.645011    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 01:00:29.119826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.119784    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"]
Oct 30 01:00:29.126510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.126337    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.138268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.138234    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqxvt\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.138636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.138612    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.138989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.138970    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.139211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.139192    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.139398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.139366    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.139605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.139587    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.139885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.139838    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.139988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.139934    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.140075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.140052    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.221917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.220228    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"]
Oct 30 01:00:29.242534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242499    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.242728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242547    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.242728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242587    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.242728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242612    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.242728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242659    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.242728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242682    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqxvt\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.243041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242761    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.243041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242816    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.243041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.242876    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.244150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.244118    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.244303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.244279    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.244454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.244431    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.244577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.244556    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.250886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.245162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.317883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.294884    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.317883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.295772    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.319198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.319165    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.322971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.321891    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqxvt\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4rlhc\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.420608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.420517    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:29.555062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.490693    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:29.555062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.547318    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:29.555062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.547380    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:29.672920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.664106    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:29.672920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:29.664171    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:30.212271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.212243    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"]
Oct 30 01:00:30.212581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.212550    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:30.212641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.212619    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:30.212676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.212649    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:30.540124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.540092    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:30.540324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.540149    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:30.640888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.640825    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:30.641242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.640913    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:30.803922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.803436    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" event={"ID":"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4","Type":"ContainerStarted","Data":"39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e"}
Oct 30 01:00:30.803922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.803508    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" event={"ID":"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4","Type":"ContainerStarted","Data":"9c8c38cc0fd7984f05fe675cd8c9d79754f934c4e60dc554b6229ed44bca8e21"}
Oct 30 01:00:30.888873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:30.888752    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podStartSLOduration=1.888730988 podStartE2EDuration="1.888730988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:30.869505726 +0000 UTC m=+1138.898815958" watchObservedRunningTime="2026-10-30 01:00:30.888730988 +0000 UTC m=+1138.918041220"
Oct 30 01:00:31.322512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.322476    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:31.493254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.493222    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:31.493605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.493582    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:31.493769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.493743    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:31.539892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.539610    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:31.539892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.539667    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:31.641954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.641521    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:31.641954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:31.641581    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:32.491797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:32.491340    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:32.491797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:32.491410    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:32.541320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:32.541287    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:32.541650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:32.541605    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:32.643958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:32.642673    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:32.643958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:32.642739    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:33.307674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.307617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:33.515352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.515315    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:33.515614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.515584    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:33.540392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.540355    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:33.541079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.541044    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:33.642727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.642634    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:33.643006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:33.642976    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:34.507870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.496265    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:34.507870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.496331    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:34.563872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.549985    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:34.563872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.550055    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:34.566991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.566948    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:34.649276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.649240    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:34.649564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.649533    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:34.888929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:34.884975    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:35.301020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.300975    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:35.499937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.499901    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:35.500211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.500183    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:35.542150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.542116    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:35.542752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.542721    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:35.641569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.641483    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 01:00:35.641871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.641817    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 01:00:35.683079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.674586    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"]
Oct 30 01:00:35.817872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.815682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:35.916927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.896931    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"]
Oct 30 01:00:35.916927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.897063    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:35.956347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:35.956315    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-153746-openshift-default-dockercfg-2jp5w\""
Oct 30 01:00:36.012774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.010121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:36.062252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.062216    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068218    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068381    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068405    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068442    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068521    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068557    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.068871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.068615    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msrz7\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.204707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.204612    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.205004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.204973    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.209486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.209369    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.209687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.209666    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.233552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.233470    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.233864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.233799    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.234151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.234072    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msrz7\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.234661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.234592    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.234863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.234636    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.255032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.254311    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.266097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.266054    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.335142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.335097    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:36.363107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.363076    4577 scope.go:122] "RemoveContainer" containerID="63d0578790f70c37dc33437e2e726b885a07bbf995a93816dde7e76311ddee2e"
Oct 30 01:00:36.364501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.364475    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.364952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.364930    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.386898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.371400    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.402439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.402411    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:00:36.459266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.459078    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.476187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.476149    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.496910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.496021    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msrz7\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:36.504885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.501419    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:36.659179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.659127    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:36.659817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.659761    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:36.956169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.956117    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/health\": context deadline exceeded"
Oct 30 01:00:36.960196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:36.960161    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 01:00:37.036135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.027072    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token\") pod \"e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") " pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:37.067420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.065441    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 01:00:37.067420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.065534    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 01:00:37.269874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.224837    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:37.369038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.368912    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:37.418022 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.417972    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:37.418222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.418047    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:37.611800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.611760    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:37.612485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.612459    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:37.612677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.612652    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:37.957127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.957089    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:37.973877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.969009    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:37.993465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.993409    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:38.008408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:37.969814    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:38.008408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.008259    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 01:00:38.160389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.160352    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:38.413871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.390591    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:38.534952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.525173    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:38.534952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.525223    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:38.696879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.693102    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 01:00:38.718474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.717635    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:38.718474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.717691    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:38.845933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.845089    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 01:00:38.845933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.845194    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:38.850182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.850159    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:38.862475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.862440    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:00:38.985228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.978360    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:38.993955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:38.988797    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 01:00:39.015005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.014971    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:39.015316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.015271    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:39.502439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.502379    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:39.518487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.511392    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:39.518903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.518816    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:39.620965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.620935    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 01:00:39.646875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.645111    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:00:39.792466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.792428    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:39.792718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.792695    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:39.792911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.792890    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:39.925688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:39.925653    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"]
Oct 30 01:00:40.089920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.072660    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:40.089920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.072936    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:00:40.089920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.072998    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:00:40.494866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.494230    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:40.494866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.494290    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:40.541447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.540967    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" event={"ID":"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0","Type":"ContainerStarted","Data":"046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"}
Oct 30 01:00:40.541447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.541138    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" event={"ID":"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0","Type":"ContainerStarted","Data":"df1f0897503736714b028fc27d6cfe68f927ffb8180831b7899cef455cb594eb"}
Oct 30 01:00:40.635921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.634645    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:40.638160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.638130    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:40.647445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.647416    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:00:40.716871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:40.707086    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podStartSLOduration=5.707063944 podStartE2EDuration="5.707063944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:40.601148732 +0000 UTC m=+1148.630458964" watchObservedRunningTime="2026-10-30 01:00:40.707063944 +0000 UTC m=+1148.736374176"
Oct 30 01:00:41.250670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.250635    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:41.258741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.258713    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:41.493868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.491649    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4rlhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:00:41.493868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.491727    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:00:41.550928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.550330    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:41.551190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.551165    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:00:41.830337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:41.830250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:42.922223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:42.922188    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:42.924493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:42.924468    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:42.925945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:42.925708    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:00:43.604903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:43.604823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:45.434685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:45.434647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:46.041797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:46.041765    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"]
Oct 30 01:00:46.073215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:46.073179    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20584efb-j6ks7"]
Oct 30 01:00:46.114225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:46.114198    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87167610-fc51-4f91-bd77-8eab85d43f6c" path="/var/lib/kubelet/pods/87167610-fc51-4f91-bd77-8eab85d43f6c/volumes"
Oct 30 01:00:47.101498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:47.101465    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:48.828625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:48.828592    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:50.489481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:50.489441    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:52.234453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:52.234413    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:53.927601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:53.927567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:54.091820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:54.090895    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"]
Oct 30 01:00:54.095905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:54.095875    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dbcf7442-6fg2p"]
Oct 30 01:00:54.112449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:54.112420    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9902b50-2d2f-454f-b663-0d1e443c3632" path="/var/lib/kubelet/pods/e9902b50-2d2f-454f-b663-0d1e443c3632/volumes"
Oct 30 01:00:55.716063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:55.716034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:57.014283 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:00:57.014253    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:00:57.618798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:57.618763    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:59.199888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.199831    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"]
Oct 30 01:00:59.206170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.206140    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.232616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.232583    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"]
Oct 30 01:00:59.320982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.320948    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:00:59.336106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336082    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336138    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336199    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336289    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336364    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336419    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336495    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336533    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.336650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.336631    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tppm\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.441616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.441560    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.442033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.442008    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.442177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.442162    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.442304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.442291    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.442436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.442423    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.442666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.442646    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tppm\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.442888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.442864    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.443058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.443044    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.443167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.443153    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.444718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.444691    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.446979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.446938    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.448321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.447761    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.448510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.448011    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.448882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.448861    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.450747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.450678    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.459153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.459128    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.492864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.490640    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tppm\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.493661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.493563    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-smbdq\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.527789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.527762    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:00:59.871174 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:00:59.871142    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73cf2e06_7295_4028_9c9c_d140fe5d7147.slice/crio-bf9013df873a7ff0f25e7547cb45dbe57b44d12937f94395fb7a9cdecd771e35 WatchSource:0}: Error finding container bf9013df873a7ff0f25e7547cb45dbe57b44d12937f94395fb7a9cdecd771e35: Status 404 returned error can't find the container with id bf9013df873a7ff0f25e7547cb45dbe57b44d12937f94395fb7a9cdecd771e35
Oct 30 01:00:59.872780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.872746    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"]
Oct 30 01:00:59.874175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.874136    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:59.874288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.874223    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:00:59.874344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:00:59.874290    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:00.814606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:00.814569    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" event={"ID":"73cf2e06-7295-4028-9c9c-d140fe5d7147","Type":"ContainerStarted","Data":"c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f"}
Oct 30 01:01:00.815177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:00.814623    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" event={"ID":"73cf2e06-7295-4028-9c9c-d140fe5d7147","Type":"ContainerStarted","Data":"bf9013df873a7ff0f25e7547cb45dbe57b44d12937f94395fb7a9cdecd771e35"}
Oct 30 01:01:00.900921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:00.900721    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podStartSLOduration=1.9007057330000001 podStartE2EDuration="1.900705733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:00.886417007 +0000 UTC m=+1168.915727263" watchObservedRunningTime="2026-10-30 01:01:00.900705733 +0000 UTC m=+1168.930015964"
Oct 30 01:01:01.172933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:01.171824    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:01.537919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:01.537891    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:01:01.538261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:01.538240    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:01.538399 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:01.538378    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:02.037884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:02.037465    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"]
Oct 30 01:01:02.051960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:02.051929    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b1077b0-vzddz"]
Oct 30 01:01:02.295928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:02.292577    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2a0b9f2-18d1-47a7-befa-2a6a6d918c78" path="/var/lib/kubelet/pods/a2a0b9f2-18d1-47a7-befa-2a6a6d918c78/volumes"
Oct 30 01:01:02.373170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:02.372732    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:01:02.530538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:02.530502    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:02.530928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:02.530897    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:03.258871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.258763    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:03.500872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.500708    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:01:03.518321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.518219    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:03.543247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.537464    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:03.543247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.537523    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:03.543247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.540204    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 01:01:03.633892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.621230    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:01:03.670879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.660796    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:03.796922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:03.796519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:04.528939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:04.528907    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:04.529453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:04.528972    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:05.123139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:05.123106    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:05.277609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:05.277573    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:05.562869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:05.541618    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:05.562869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:05.541686    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:05.862655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:05.862566    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:06.528544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:06.528511    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:06.528756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:06.528583    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:07.528729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:07.528699    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:07.529189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:07.528752    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:07.621161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:07.621125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:08.529142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:08.529110    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:01:08.529608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:08.529169    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:01:09.328377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:09.328347    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:09.898721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:09.898689    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:01:09.899213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:09.899188    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:01:09.899828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:09.899804    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:01:11.115881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:11.115836    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:12.927543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:12.927495    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:14.701429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:14.701394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:16.517915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:16.517877    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:16.750346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:16.746480    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:17.379929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.379898    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"]
Oct 30 01:01:17.404149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.404120    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.408241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.408171    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"]
Oct 30 01:01:17.441872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.428368    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6482ae-openshift-default-dockercfg-t579f\""
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.468504    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470412    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470456    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470553    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470695    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470737    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9trnj\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470771    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470799    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.473700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.470826    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.474612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.474584    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.655774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.655683    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.676046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.676006    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.676459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.676415    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.676881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.676857    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.677113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.677072    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9trnj\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.678323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.678304    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.678504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.678484    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.678660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.678644    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.678986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.678967    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.736869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.683096    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.739727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.668032    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.739987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.694158    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.740198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.703255    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.740314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.703480    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:17.913930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:17.906982    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:18.194230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.194138    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9trnj\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:18.535868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.512598    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:18.636866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.528064    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy\") pod \"e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") " pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:18.656871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.546123    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:01:18.680671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.553188    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"]
Oct 30 01:01:18.703003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.702977    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ddddd62b-cnp5g"]
Oct 30 01:01:18.703225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.687706    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:18.717551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.717507    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:18.986871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:18.982345    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:19.330768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.330703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:19.387001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.386970    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"]
Oct 30 01:01:19.452693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.452554    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:19.479927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.479893    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"]
Oct 30 01:01:19.480167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.480024    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.511382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.511332    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-153746-serving-cert\""
Oct 30 01:01:19.543577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.543052    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.543577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.543181    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.543577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.543213    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.543577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.543291    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d7kz\" (UniqueName: \"kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.644543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.644513    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.644609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.644560    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.644646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.644635    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d7kz\" (UniqueName: \"kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.644811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.644793    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.646539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.645949    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.649455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.649374    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.651138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.651094    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.654443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.654335    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"]
Oct 30 01:01:19.657352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.657318    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:19.657754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.657412    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:19.657754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.657453    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:19.672924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.672904    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d7kz\" (UniqueName: \"kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz\") pod \"maas-api-e2e-api-b-153746-64b89749fb-zxn5q\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:19.865866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:19.856097    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:20.280924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.192722    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:20.326998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.326958    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b09c1d10-4db7-497f-b101-1242c8b02fa2" path="/var/lib/kubelet/pods/b09c1d10-4db7-497f-b101-1242c8b02fa2/volumes"
Oct 30 01:01:20.328806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.327489    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"]
Oct 30 01:01:20.440551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.440508    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q" event={"ID":"d5be7611-9bf8-46c0-881a-96611ebfb01d","Type":"ContainerStarted","Data":"ca28c190b1eaa57cf76693d9a387e6c0d8ace1d67847383c3e9bb9df2b51ff87"}
Oct 30 01:01:20.442493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.442463    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" event={"ID":"5f849b4b-9407-4325-bb99-6fac3dcc042f","Type":"ContainerStarted","Data":"ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"}
Oct 30 01:01:20.442667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.442544    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" event={"ID":"5f849b4b-9407-4325-bb99-6fac3dcc042f","Type":"ContainerStarted","Data":"797025e2d412975f85f8ca908c2c144a735be8c9644c20a1c7c76afe6f6e0d68"}
Oct 30 01:01:20.706922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.706766    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:20.731368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.726233    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:20.781483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.781432    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podStartSLOduration=3.781419287 podStartE2EDuration="3.781419287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:20.512523366 +0000 UTC m=+1188.541833597" watchObservedRunningTime="2026-10-30 01:01:20.781419287 +0000 UTC m=+1188.810729518"
Oct 30 01:01:20.812555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:20.812316    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:21.457173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:21.457140    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q" event={"ID":"d5be7611-9bf8-46c0-881a-96611ebfb01d","Type":"ContainerStarted","Data":"f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e"}
Oct 30 01:01:21.457362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:21.457302    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:21.458196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:21.458171    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:01:21.505001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:21.504941    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q" podStartSLOduration=3.038869854 podStartE2EDuration="3.504921678s" totalImagesPullingTime="466.051824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:21.504021579 +0000 UTC m=+1189.533331828" watchObservedRunningTime="2026-10-30 01:01:21.504921678 +0000 UTC m=+1189.534231911"
Oct 30 01:01:22.474774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:22.474728    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:22.727764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:22.727684    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:22.984657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:22.984621    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:23.117282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:23.117240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:24.661538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:24.661502    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:24.793738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:24.793700    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:26.040301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:26.040269    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"]
Oct 30 01:01:26.071981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:26.071951    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ff0379f8-lxrlc"]
Oct 30 01:01:26.125359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:26.125270    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bd748a7-8da3-44d6-b26c-d3f3eb034c99" path="/var/lib/kubelet/pods/0bd748a7-8da3-44d6-b26c-d3f3eb034c99/volumes"
Oct 30 01:01:26.494322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:26.494280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:27.015341 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:01:27.015306    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:01:28.161698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:28.161667    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:28.519616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:28.519591    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:01:29.376594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:29.376562    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:30.001292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:30.001260    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:31.795036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:31.794992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:33.502427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:33.502394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:34.301729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:34.301698    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:35.234935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:35.234890    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:35.359356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:35.359317    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:35.769091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:35.769051    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:37.031692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:37.031632    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:38.261180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.261131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:38.764858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.764825    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"]
Oct 30 01:01:38.770972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.770952    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.792952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.792912    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"]
Oct 30 01:01:38.798836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.798805    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6482ae-serving-cert\""
Oct 30 01:01:38.799328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.799066    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.799328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.799203    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.799328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.799247    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6xbg\" (UniqueName: \"kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.799328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.799303    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.860703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.860674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:38.900447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.900418    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.900659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.900465    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.900659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.900492    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6xbg\" (UniqueName: \"kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.900659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.900545    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.900659 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:01:38.900599    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-serving-cert: secret "maas-api-e2e-api-a-6482ae-serving-cert" not found
Oct 30 01:01:38.900828 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:01:38.900669    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls podName:83b82600-a7c0-4ab2-bce1-e36c3709e16b nodeName:}" failed. No retries permitted until 2026-10-30 01:01:39.4006467 +0000 UTC m=+1207.429956915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls") pod "maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" (UID: "83b82600-a7c0-4ab2-bce1-e36c3709e16b") : secret "maas-api-e2e-api-a-6482ae-serving-cert" not found
Oct 30 01:01:38.901283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.901243    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.903227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.903206    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.930887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.930839    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6xbg\" (UniqueName: \"kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:38.971371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:38.971336    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:39.139244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:39.139140    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:39.412827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:39.412748    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:39.415347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:39.415324    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\") pod \"maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:39.687707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:39.687618    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:40.239590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:40.227286    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"]
Oct 30 01:01:40.242223 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:01:40.241137    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b82600_a7c0_4ab2_bce1_e36c3709e16b.slice/crio-c32977c9b5a26d19a14fd8e35489966b7c4055218e7a05925b5aad2f0864e49d WatchSource:0}: Error finding container c32977c9b5a26d19a14fd8e35489966b7c4055218e7a05925b5aad2f0864e49d: Status 404 returned error can't find the container with id c32977c9b5a26d19a14fd8e35489966b7c4055218e7a05925b5aad2f0864e49d
Oct 30 01:01:40.726437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:40.726399    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:40.820953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:40.818535    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" event={"ID":"83b82600-a7c0-4ab2-bce1-e36c3709e16b","Type":"ContainerStarted","Data":"1ac822b5682a9492b21b6d52258c7e7d523cfedc6f1cf3d654fe3338113f9a51"}
Oct 30 01:01:40.820953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:40.818660    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" event={"ID":"83b82600-a7c0-4ab2-bce1-e36c3709e16b","Type":"ContainerStarted","Data":"c32977c9b5a26d19a14fd8e35489966b7c4055218e7a05925b5aad2f0864e49d"}
Oct 30 01:01:40.820953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:40.820668    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:40.961979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:40.959786    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" podStartSLOduration=2.630527303 podStartE2EDuration="2.959768966s" totalImagesPullingTime="329.241663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:40.954235168 +0000 UTC m=+1208.983545402" watchObservedRunningTime="2026-10-30 01:01:40.959768966 +0000 UTC m=+1208.989079191"
Oct 30 01:01:42.622204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:42.622096    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:44.594869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:44.587094    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:45.059867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:45.059483    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"]
Oct 30 01:01:45.095432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:45.095385    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dd465542-pvhz4"]
Oct 30 01:01:46.194871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:46.180746    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22df6c37-ebc8-4e39-92a3-afb004f358f8" path="/var/lib/kubelet/pods/22df6c37-ebc8-4e39-92a3-afb004f358f8/volumes"
Oct 30 01:01:46.718883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:46.714511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:47.488869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:47.487964    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:01:47.502870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:47.493211    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:01:47.966275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:47.966159    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:01:48.524395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:48.524356    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 01:01:48.532327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:48.532088    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:01:48.630467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:48.630439    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:01:48.749610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:48.749579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:50.515183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:50.515143    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:51.938598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:51.938505    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"]
Oct 30 01:01:51.977610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:51.977571    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"]
Oct 30 01:01:52.070362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:52.070322    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:52.196052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:52.195901    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:52.417615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:52.417576    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:54.327497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:54.327464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:55.031446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:55.031408    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"]
Oct 30 01:01:55.048553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:55.048524    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2d93b59-tzzks"]
Oct 30 01:01:55.695816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:55.695775    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:56.023903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.023864    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:56.191871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.188037    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2299a7b9-1527-4349-927b-0df2e359c37a" path="/var/lib/kubelet/pods/2299a7b9-1527-4349-927b-0df2e359c37a/volumes"
Oct 30 01:01:56.552068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.552026    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:01:56.552290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.552082    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:01:56.814215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.814131    4577 generic.go:356] "Generic (PLEG): container finished" podID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerID="046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100" exitCode=0
Oct 30 01:01:56.814215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.814200    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" event={"ID":"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0","Type":"ContainerDied","Data":"046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"}
Oct 30 01:01:56.814706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.814690    4577 scope.go:122] "RemoveContainer" containerID="046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"
Oct 30 01:01:56.816310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.816286    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:56.816372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.816351    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:56.816413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.816383    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:01:56.939150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.939083    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" podUID="9e69b637-61db-439b-a9d5-5878afeed344" containerName="payload-processing" containerID="cri-o://ac7c318df9fa778e688a7858600e5f18d1fdc554ad9a0a7af77b801f53ccee24" gracePeriod=25
Oct 30 01:01:56.978682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:56.978629    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerName="payload-pre-processing" containerID="cri-o://4e34bd8bb7fdb7026c60a2b35a9a226bd84dbb5e9244a0a4f30b4fbd28114f84" gracePeriod=25
Oct 30 01:01:57.033425 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:01:57.033378    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:01:57.709129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:57.709095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:57.853868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:57.843779    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" event={"ID":"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0","Type":"ContainerStarted","Data":"325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759"}
Oct 30 01:01:58.255251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:58.255218    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:01:58.276008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:58.263955    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:01:58.276230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:58.276038    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:01:58.799766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:58.799725    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:01:59.264439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:59.264402    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:01:59.265114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:59.265081    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:01:59.553888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:59.553761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:01:59.820316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:01:59.820223    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" podUID="9e69b637-61db-439b-a9d5-5878afeed344" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:02:00.114439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:00.114354    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:00.224415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:00.224382    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:00.257877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:00.252267    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:02:00.257877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:00.252330    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:02:01.251209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.251174    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:02:01.251821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.251791    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:02:01.409885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.408880    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:01.544930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.544054    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"]
Oct 30 01:02:01.586535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.586490    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"]
Oct 30 01:02:01.637659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.637622    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:01.783303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:01.783243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:02.275249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:02.275213    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:02:02.292744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:02.292706    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:02:03.020806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:03.020764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:03.250330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:03.250288    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused" start-of-body=
Oct 30 01:02:03.250634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:03.250596    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.120:15021/healthz/ready\": dial tcp 10.134.0.120:15021: connect: connection refused"
Oct 30 01:02:04.279027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:04.278391    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:02:04.280041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:04.280018    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:02:04.281122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:04.281085    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:02:04.792250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:04.792221    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:06.548903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:06.547948    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj" podUID="1dd93e79-08f8-4c30-b0f0-120d370a93aa" containerName="payload-processing" containerID="cri-o://55b93fa0291a262f4f06991125b51642697d1469db11fdd98fb953a6b5e1545e" gracePeriod=25
Oct 30 01:02:06.587268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:06.587229    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv" podUID="f81da315-c2a3-425d-ad89-32d89283d3e3" containerName="payload-pre-processing" containerID="cri-o://22bee88bbb98fcf57ab11af47701d38ee16e17cd86440d56d1f4a998c551928e" gracePeriod=25
Oct 30 01:02:06.705897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:06.705576    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:07.113209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.113177    4577 generic.go:356] "Generic (PLEG): container finished" podID="f81da315-c2a3-425d-ad89-32d89283d3e3" containerID="22bee88bbb98fcf57ab11af47701d38ee16e17cd86440d56d1f4a998c551928e" exitCode=0
Oct 30 01:02:07.113475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.113424    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv" event={"ID":"f81da315-c2a3-425d-ad89-32d89283d3e3","Type":"ContainerDied","Data":"22bee88bbb98fcf57ab11af47701d38ee16e17cd86440d56d1f4a998c551928e"}
Oct 30 01:02:07.115336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.115311    4577 generic.go:356] "Generic (PLEG): container finished" podID="1dd93e79-08f8-4c30-b0f0-120d370a93aa" containerID="55b93fa0291a262f4f06991125b51642697d1469db11fdd98fb953a6b5e1545e" exitCode=0
Oct 30 01:02:07.115455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.115441    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj" event={"ID":"1dd93e79-08f8-4c30-b0f0-120d370a93aa","Type":"ContainerDied","Data":"55b93fa0291a262f4f06991125b51642697d1469db11fdd98fb953a6b5e1545e"}
Oct 30 01:02:07.138096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.138025    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 01:02:07.237807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.237775    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume\") pod \"f81da315-c2a3-425d-ad89-32d89283d3e3\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") "
Oct 30 01:02:07.238051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.237879    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x\" (UniqueName: \"kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x\") pod \"f81da315-c2a3-425d-ad89-32d89283d3e3\" (UID: \"f81da315-c2a3-425d-ad89-32d89283d3e3\") "
Oct 30 01:02:07.238548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.238526    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 01:02:07.239384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.239358    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume" pod "f81da315-c2a3-425d-ad89-32d89283d3e3" (UID: "f81da315-c2a3-425d-ad89-32d89283d3e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:02:07.282087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.282044    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x" pod "f81da315-c2a3-425d-ad89-32d89283d3e3" (UID: "f81da315-c2a3-425d-ad89-32d89283d3e3"). InnerVolumeSpecName "kube-api-access-fr25x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:02:07.339663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.339623    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume\") pod \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") "
Oct 30 01:02:07.339946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.339919    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9\" (UniqueName: \"kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9\") pod \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\" (UID: \"1dd93e79-08f8-4c30-b0f0-120d370a93aa\") "
Oct 30 01:02:07.340052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.340025    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume" pod "1dd93e79-08f8-4c30-b0f0-120d370a93aa" (UID: "1dd93e79-08f8-4c30-b0f0-120d370a93aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:02:07.340632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.340607    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f81da315-c2a3-425d-ad89-32d89283d3e3-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:07.340708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.340638    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1dd93e79-08f8-4c30-b0f0-120d370a93aa-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:07.340708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.340653    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr25x\" (UniqueName: \"kubernetes.io/projected/f81da315-c2a3-425d-ad89-32d89283d3e3-kube-api-access-fr25x\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:07.342467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.342442    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9" pod "1dd93e79-08f8-4c30-b0f0-120d370a93aa" (UID: "1dd93e79-08f8-4c30-b0f0-120d370a93aa"). InnerVolumeSpecName "kube-api-access-vmwl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:02:07.441896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:07.441792    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmwl9\" (UniqueName: \"kubernetes.io/projected/1dd93e79-08f8-4c30-b0f0-120d370a93aa-kube-api-access-vmwl9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:08.124497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.124472    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"
Oct 30 01:02:08.124869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.124498    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj" event={"ID":"1dd93e79-08f8-4c30-b0f0-120d370a93aa","Type":"ContainerDied","Data":"71b51d6c5751c89f932ea047c1eb6ff08f60833447e7b3ba7a01253e82de4d35"}
Oct 30 01:02:08.124869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.124561    4577 scope.go:122] "RemoveContainer" containerID="55b93fa0291a262f4f06991125b51642697d1469db11fdd98fb953a6b5e1545e"
Oct 30 01:02:08.126298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.126277    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"
Oct 30 01:02:08.126414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.126325    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv" event={"ID":"f81da315-c2a3-425d-ad89-32d89283d3e3","Type":"ContainerDied","Data":"c749022a840a252e8f52a93b90e50e7f6032d24ed9d3b2588aacda0680098779"}
Oct 30 01:02:08.137578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.137555    4577 scope.go:122] "RemoveContainer" containerID="22bee88bbb98fcf57ab11af47701d38ee16e17cd86440d56d1f4a998c551928e"
Oct 30 01:02:08.187437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.187375    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"]
Oct 30 01:02:08.198084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.198053    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-52b947-77fb7f7f44-m55jj"]
Oct 30 01:02:08.230511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.230478    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"]
Oct 30 01:02:08.236524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.236494    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-52b947-6ddb7df9d7-8k4xv"]
Oct 30 01:02:08.397697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.397584    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:08.519890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.518240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:08.799982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:08.799937    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:02:09.142344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:09.142251    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:09.257570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:09.257515    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:09.809755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:09.809710    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" podUID="9e69b637-61db-439b-a9d5-5878afeed344" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:02:10.113744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:10.113652    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1dd93e79-08f8-4c30-b0f0-120d370a93aa" path="/var/lib/kubelet/pods/1dd93e79-08f8-4c30-b0f0-120d370a93aa/volumes"
Oct 30 01:02:10.114353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:10.114299    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f81da315-c2a3-425d-ad89-32d89283d3e3" path="/var/lib/kubelet/pods/f81da315-c2a3-425d-ad89-32d89283d3e3/volumes"
Oct 30 01:02:10.266096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:10.266061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:12.004403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:12.004363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:13.795298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:13.795250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:15.559179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:15.559148    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:16.223327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:16.223258    4577 generic.go:356] "Generic (PLEG): container finished" podID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerID="ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab" exitCode=0
Oct 30 01:02:16.223558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:16.223337    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" event={"ID":"5f849b4b-9407-4325-bb99-6fac3dcc042f","Type":"ContainerDied","Data":"ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"}
Oct 30 01:02:16.224335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:16.224313    4577 scope.go:122] "RemoveContainer" containerID="ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"
Oct 30 01:02:16.226065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:16.226036    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:02:16.226172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:16.226112    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:02:16.226172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:16.226148    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:02:17.237160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.237124    4577 generic.go:356] "Generic (PLEG): container finished" podID="9e69b637-61db-439b-a9d5-5878afeed344" containerID="ac7c318df9fa778e688a7858600e5f18d1fdc554ad9a0a7af77b801f53ccee24" exitCode=0
Oct 30 01:02:17.237667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.237242    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" event={"ID":"9e69b637-61db-439b-a9d5-5878afeed344","Type":"ContainerDied","Data":"ac7c318df9fa778e688a7858600e5f18d1fdc554ad9a0a7af77b801f53ccee24"}
Oct 30 01:02:17.239583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.239315    4577 generic.go:356] "Generic (PLEG): container finished" podID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerID="4e34bd8bb7fdb7026c60a2b35a9a226bd84dbb5e9244a0a4f30b4fbd28114f84" exitCode=0
Oct 30 01:02:17.239583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.239524    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" event={"ID":"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4","Type":"ContainerDied","Data":"4e34bd8bb7fdb7026c60a2b35a9a226bd84dbb5e9244a0a4f30b4fbd28114f84"}
Oct 30 01:02:17.241860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.241823    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" event={"ID":"5f849b4b-9407-4325-bb99-6fac3dcc042f","Type":"ContainerStarted","Data":"c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541"}
Oct 30 01:02:17.306879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.306813    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:17.336153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.336117    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 01:02:17.470875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.467061    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume\") pod \"9e69b637-61db-439b-a9d5-5878afeed344\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") "
Oct 30 01:02:17.470875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.467135    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q\" (UniqueName: \"kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q\") pod \"9e69b637-61db-439b-a9d5-5878afeed344\" (UID: \"9e69b637-61db-439b-a9d5-5878afeed344\") "
Oct 30 01:02:17.477899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.477838    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume" pod "9e69b637-61db-439b-a9d5-5878afeed344" (UID: "9e69b637-61db-439b-a9d5-5878afeed344"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:02:17.483529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.483497    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e69b637-61db-439b-a9d5-5878afeed344-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:17.537941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.527403    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q" pod "9e69b637-61db-439b-a9d5-5878afeed344" (UID: "9e69b637-61db-439b-a9d5-5878afeed344"). InnerVolumeSpecName "kube-api-access-khb7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:02:17.590187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.590159    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khb7q\" (UniqueName: \"kubernetes.io/projected/9e69b637-61db-439b-a9d5-5878afeed344-kube-api-access-khb7q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:17.599950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.599893    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 01:02:17.687919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.687892    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:02:17.688834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.688803    4577 patch_prober.go:29] interesting pod/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 01:02:17.688961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.688894    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 01:02:17.706796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.706673    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2\" (UniqueName: \"kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2\") pod \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") "
Oct 30 01:02:17.707275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.707247    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume\") pod \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\" (UID: \"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4\") "
Oct 30 01:02:17.709149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.709115    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume" pod "655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" (UID: "655c9ee8-6c8a-42fe-a094-cd0c9a4226c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:02:17.710407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.710381    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:17.714069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.714034    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2" pod "655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" (UID: "655c9ee8-6c8a-42fe-a094-cd0c9a4226c4"). InnerVolumeSpecName "kube-api-access-lv2c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:02:17.811689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:17.811604    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv2c2\" (UniqueName: \"kubernetes.io/projected/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4-kube-api-access-lv2c2\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:18.279087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.279052    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"
Oct 30 01:02:18.279087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.279067    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg" event={"ID":"9e69b637-61db-439b-a9d5-5878afeed344","Type":"ContainerDied","Data":"5318f531d89f9dbe5cf9d3fc486b430d716ff5f6e32a5c1f0d5c5ce43d3662b6"}
Oct 30 01:02:18.279658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.279148    4577 scope.go:122] "RemoveContainer" containerID="ac7c318df9fa778e688a7858600e5f18d1fdc554ad9a0a7af77b801f53ccee24"
Oct 30 01:02:18.281060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.281027    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"
Oct 30 01:02:18.281209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.281061    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm" event={"ID":"655c9ee8-6c8a-42fe-a094-cd0c9a4226c4","Type":"ContainerDied","Data":"1aca5dc4c05737f4110809695d6153fb8341ababf5efbc81c3579bf103e5ced4"}
Oct 30 01:02:18.299045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.298791    4577 scope.go:122] "RemoveContainer" containerID="4e34bd8bb7fdb7026c60a2b35a9a226bd84dbb5e9244a0a4f30b4fbd28114f84"
Oct 30 01:02:18.341885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.341839    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"]
Oct 30 01:02:18.374927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.374434    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8363a40b-777cf99df8-cvtmg"]
Oct 30 01:02:18.409807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.409774    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"]
Oct 30 01:02:18.414257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.414235    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8363a40b-756859fd87-rmnjm"]
Oct 30 01:02:18.688445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.688352    4577 patch_prober.go:29] interesting pod/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 01:02:18.688445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:18.688418    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 01:02:19.125828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.125783    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:19.603956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.603917    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"]
Oct 30 01:02:19.604716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604699    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1dd93e79-08f8-4c30-b0f0-120d370a93aa" containerName="payload-processing"
Oct 30 01:02:19.604760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604718    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dd93e79-08f8-4c30-b0f0-120d370a93aa" containerName="payload-processing"
Oct 30 01:02:19.604799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604766    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f81da315-c2a3-425d-ad89-32d89283d3e3" containerName="payload-pre-processing"
Oct 30 01:02:19.604799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604772    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f81da315-c2a3-425d-ad89-32d89283d3e3" containerName="payload-pre-processing"
Oct 30 01:02:19.604799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604784    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9e69b637-61db-439b-a9d5-5878afeed344" containerName="payload-processing"
Oct 30 01:02:19.604799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604789    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e69b637-61db-439b-a9d5-5878afeed344" containerName="payload-processing"
Oct 30 01:02:19.604969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604830    4577 cpu_manager.go:414] "containerMap: removing container" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerName="payload-pre-processing"
Oct 30 01:02:19.604969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604836    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerName="payload-pre-processing"
Oct 30 01:02:19.605035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.604992    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dd93e79-08f8-4c30-b0f0-120d370a93aa" containerName="payload-processing"
Oct 30 01:02:19.605035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.605010    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f81da315-c2a3-425d-ad89-32d89283d3e3" containerName="payload-pre-processing"
Oct 30 01:02:19.605035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.605018    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e69b637-61db-439b-a9d5-5878afeed344" containerName="payload-processing"
Oct 30 01:02:19.605035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.605028    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" containerName="payload-pre-processing"
Oct 30 01:02:19.610884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.608195    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.643621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.638798    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9a8d85-openshift-default-dockercfg-x8rls\""
Oct 30 01:02:19.643621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.639214    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"]
Oct 30 01:02:19.678201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.661608    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:19.688691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.688655    4577 patch_prober.go:29] interesting pod/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 01:02:19.689001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.688968    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 01:02:19.786874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.762285    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.787253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.787220    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.796084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.796050    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.796424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.796393    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.796651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.796631    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.796912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.796892    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.797087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.797070    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.797277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.797256    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.805044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.805013    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtjk7\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.805519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.791758    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:19.912770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.912677    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.913545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.913518    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.914209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.914105    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.914408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.914179    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.915051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.915033    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.915731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.915713    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtjk7\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.916314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.916288    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.920018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.919999    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.920539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.920520    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.925603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.925584    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.930454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.916687    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.930608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.925523    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.930800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.915431    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.930960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.930148    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.943871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.939290    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.944226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.944168    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.969873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.960028    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtjk7\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:19.969873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:19.962771    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:20.145870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.145275    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="655c9ee8-6c8a-42fe-a094-cd0c9a4226c4" path="/var/lib/kubelet/pods/655c9ee8-6c8a-42fe-a094-cd0c9a4226c4/volumes"
Oct 30 01:02:20.157869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.157126    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e69b637-61db-439b-a9d5-5878afeed344" path="/var/lib/kubelet/pods/9e69b637-61db-439b-a9d5-5878afeed344/volumes"
Oct 30 01:02:20.252873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.252351    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:20.314281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.314247    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:20.604969 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:02:20.604941    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bad413b_e468_498a_8343_29be621eeeae.slice/crio-4baf1d78af1e89de951c906f55ac9cdfaaa0433ca73226bc22543c09466bdc1a WatchSource:0}: Error finding container 4baf1d78af1e89de951c906f55ac9cdfaaa0433ca73226bc22543c09466bdc1a: Status 404 returned error can't find the container with id 4baf1d78af1e89de951c906f55ac9cdfaaa0433ca73226bc22543c09466bdc1a
Oct 30 01:02:20.605298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.604994    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"]
Oct 30 01:02:20.607940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.607907    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:02:20.608080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.607980    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:02:20.608080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.608012    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:02:20.688404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.688373    4577 patch_prober.go:29] interesting pod/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 01:02:20.688562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.688426    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 01:02:20.991997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:20.991966    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:21.325454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:21.325367    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" event={"ID":"4bad413b-e468-498a-8343-29be621eeeae","Type":"ContainerStarted","Data":"8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"}
Oct 30 01:02:21.325454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:21.325434    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" event={"ID":"4bad413b-e468-498a-8343-29be621eeeae","Type":"ContainerStarted","Data":"4baf1d78af1e89de951c906f55ac9cdfaaa0433ca73226bc22543c09466bdc1a"}
Oct 30 01:02:21.381940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:21.381886    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podStartSLOduration=2.381869682 podStartE2EDuration="2.381869682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:02:21.373403198 +0000 UTC m=+1249.402713441" watchObservedRunningTime="2026-10-30 01:02:21.381869682 +0000 UTC m=+1249.411179912"
Oct 30 01:02:21.464869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:21.460925    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:02:21.688502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:21.688417    4577 patch_prober.go:29] interesting pod/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 01:02:21.688502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:21.688471    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 01:02:22.255871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.255327    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:22.281908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.281877    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:22.334346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.334313    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:22.335330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.335311    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:02:22.696626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.696538    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:02:22.698383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.698350    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:02:22.878999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:22.878958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:23.834171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:23.834134    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:23.948752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:23.948715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:24.198966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:24.198887    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:24.724160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:24.724126    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:24.906347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:24.906314    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:25.587311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:25.587280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:25.702278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:25.702241    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:26.478266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:26.478234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:27.000615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:27.000580    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:27.120346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:27.120311    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:27.138594 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:02:27.138557    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:02:27.252259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:27.252167    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:29.057895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:29.057862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:31.003340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:31.003300    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:32.693746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:32.693658    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:34.386956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:34.386919    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:36.090392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:36.090354    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:37.548620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:37.548580    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"]
Oct 30 01:02:37.549981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:37.548876    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn" podUID="4c060341-33b8-418b-945d-e3af71509aa4" containerName="maas-api" containerID="cri-o://5ab129f340236d2657e578a803a4f789e5996d266f56cfc68aa2dea64130fd75" gracePeriod=30
Oct 30 01:02:37.616938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:37.616904    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:37.686324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:37.686288    4577 generic.go:356] "Generic (PLEG): container finished" podID="4c060341-33b8-418b-945d-e3af71509aa4" containerID="5ab129f340236d2657e578a803a4f789e5996d266f56cfc68aa2dea64130fd75" exitCode=0
Oct 30 01:02:37.686488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:37.686357    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn" event={"ID":"4c060341-33b8-418b-945d-e3af71509aa4","Type":"ContainerDied","Data":"5ab129f340236d2657e578a803a4f789e5996d266f56cfc68aa2dea64130fd75"}
Oct 30 01:02:38.157988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.151707    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:38.283130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.282685    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:38.313667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.299038    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:02:38.393914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.392538    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs\") pod \"4c060341-33b8-418b-945d-e3af71509aa4\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") "
Oct 30 01:02:38.393914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.392601    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca\") pod \"4c060341-33b8-418b-945d-e3af71509aa4\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") "
Oct 30 01:02:38.393914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.392674    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls\") pod \"4c060341-33b8-418b-945d-e3af71509aa4\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") "
Oct 30 01:02:38.393914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.392758    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp\" (UniqueName: \"kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp\") pod \"4c060341-33b8-418b-945d-e3af71509aa4\" (UID: \"4c060341-33b8-418b-945d-e3af71509aa4\") "
Oct 30 01:02:38.436739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.436083    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca" pod "4c060341-33b8-418b-945d-e3af71509aa4" (UID: "4c060341-33b8-418b-945d-e3af71509aa4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:02:38.444675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.444647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:38.465882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.462169    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls" pod "4c060341-33b8-418b-945d-e3af71509aa4" (UID: "4c060341-33b8-418b-945d-e3af71509aa4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:02:38.470438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.470405    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp" pod "4c060341-33b8-418b-945d-e3af71509aa4" (UID: "4c060341-33b8-418b-945d-e3af71509aa4"). InnerVolumeSpecName "kube-api-access-gsxlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:02:38.472466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.472426    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs" pod "4c060341-33b8-418b-945d-e3af71509aa4" (UID: "4c060341-33b8-418b-945d-e3af71509aa4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:02:38.495768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.495737    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsxlp\" (UniqueName: \"kubernetes.io/projected/4c060341-33b8-418b-945d-e3af71509aa4-kube-api-access-gsxlp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:38.495768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.495771    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:38.496047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.495786    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c060341-33b8-418b-945d-e3af71509aa4-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:38.496047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.495794    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c060341-33b8-418b-945d-e3af71509aa4-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:02:38.583411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.583381    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:38.727786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.727545    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:38.735886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.733033    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn" event={"ID":"4c060341-33b8-418b-945d-e3af71509aa4","Type":"ContainerDied","Data":"c9ac745d4017228bdb86fc8daaa61283e7bdef69f3f5cfe0f075e982a5a61a3d"}
Oct 30 01:02:38.735886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.733105    4577 scope.go:122] "RemoveContainer" containerID="5ab129f340236d2657e578a803a4f789e5996d266f56cfc68aa2dea64130fd75"
Oct 30 01:02:38.735886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.733266    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"
Oct 30 01:02:38.840101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.840078    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"]
Oct 30 01:02:38.848892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.847537    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-52b947-dbd447ddb-jsccn"]
Oct 30 01:02:38.848892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.848363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:38.898648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.898617    4577 scope.go:122] "RemoveContainer" containerID="b9be24b9686f1e178c04e943f63b2e406bad33f0d1b53a6e566632d7cc4f96c8"
Oct 30 01:02:38.909963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.909943    4577 scope.go:122] "RemoveContainer" containerID="bed34299f49a3ba7a426cf380d5b8bffe5b158ec46e324e13242f1931ba133e1"
Oct 30 01:02:38.920167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.920147    4577 scope.go:122] "RemoveContainer" containerID="7189c4bdc56b0fec50739dcf419709cc8348b74e40dbc9e6ad1b2a4289657520"
Oct 30 01:02:38.930084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.930063    4577 scope.go:122] "RemoveContainer" containerID="9f9638937ce0cf46f88926260ea400ca9d8f354e6355b6752dd3621bcb2825cc"
Oct 30 01:02:38.939506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:38.939483    4577 scope.go:122] "RemoveContainer" containerID="4c83d4f0834aa6fe9147decf6c9c9da4b9f0682081c5e085922411ff516c1905"
Oct 30 01:02:40.113234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:40.113200    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c060341-33b8-418b-945d-e3af71509aa4" path="/var/lib/kubelet/pods/4c060341-33b8-418b-945d-e3af71509aa4/volumes"
Oct 30 01:02:40.237086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:40.237055    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:40.418829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:40.418741    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:40.537571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:40.537540    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:41.994118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:41.994086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:43.723188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:43.723159    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:45.513142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:45.513111    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:47.216282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:47.216250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:48.787433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:48.787396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:48.905265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:48.905234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:50.609486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:50.609445    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:52.137095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:52.135397    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:52.263581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:52.263327    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:52.534138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:52.510472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:54.750873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.735236    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"]
Oct 30 01:02:54.750873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.736329    4577 cpu_manager.go:414] "containerMap: removing container" podUID="4c060341-33b8-418b-945d-e3af71509aa4" containerName="maas-api"
Oct 30 01:02:54.750873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.736351    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c060341-33b8-418b-945d-e3af71509aa4" containerName="maas-api"
Oct 30 01:02:54.750873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.736574    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c060341-33b8-418b-945d-e3af71509aa4" containerName="maas-api"
Oct 30 01:02:54.750873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.742330    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:54.775941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.769195    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9b2c3f-serving-cert\""
Oct 30 01:02:55.058044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:54.968716    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.078554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.076967    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.079786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.079739    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brz2q\" (UniqueName: \"kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.080025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.080007    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.080425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.068427    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"]
Oct 30 01:02:55.080638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.022510    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:55.186866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.186812    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.237669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.230663    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.237669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.230836    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.237669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.230969    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brz2q\" (UniqueName: \"kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.237669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.187814    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.237669 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:02:55.231923    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-serving-cert: secret "maas-api-e2e-api-a-9b2c3f-serving-cert" not found
Oct 30 01:02:55.237669 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:02:55.231998    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls podName:a9701959-bf30-41cd-b0a7-b156593ff26b nodeName:}" failed. No retries permitted until 2026-10-30 01:02:55.731977143 +0000 UTC m=+1283.761287369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls") pod "maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" (UID: "a9701959-bf30-41cd-b0a7-b156593ff26b") : secret "maas-api-e2e-api-a-9b2c3f-serving-cert" not found
Oct 30 01:02:55.267888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.267796    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brz2q\" (UniqueName: \"kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.278053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.278014    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:55.329695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.329615    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.489028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.488975    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:55.535067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.535026    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:55.547322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.547291    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-lqpbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:02:55.547628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.547598    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:55.612927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.610332    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:55.641144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.641102    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 01:02:55.666115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.654959    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:55.771208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.771173    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:55.786213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.786171    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.822882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.818432    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\") pod \"maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:55.891377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.891260    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:55.970176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:55.970141    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:02:56.037970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.034968    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-smbdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:02:56.037970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.035029    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:56.201958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.195988    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:56.498925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.489730    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:56.538017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.537981    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:02:56.706872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.685232    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:56.706872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.685826    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:02:56.943962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:56.940154    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:57.503059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.503013    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 01:02:57.582452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.578834    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded"
Oct 30 01:02:57.595110 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:02:57.595075    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:02:57.628169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.628140    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:02:57.695613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.659928    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:57.695613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.695193    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:02:57.743871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.736987    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"]
Oct 30 01:02:57.957873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:57.945637    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:02:58.548424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:58.532971    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:58.998870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:58.984641    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" event={"ID":"a9701959-bf30-41cd-b0a7-b156593ff26b","Type":"ContainerStarted","Data":"12ea449a923d686a290374e330495384b338017538d07b2af16b2de78c696bd6"}
Oct 30 01:02:59.326942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:59.326906    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:59.831371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:59.831334    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:02:59.996571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:59.996538    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" event={"ID":"a9701959-bf30-41cd-b0a7-b156593ff26b","Type":"ContainerStarted","Data":"3042da501ca42027f33ddf5b430c77ad1adf4618193c1602d3d22c93c863a138"}
Oct 30 01:02:59.996756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:02:59.996679    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:03:00.079154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:00.079087    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" podStartSLOduration=5.397073948 podStartE2EDuration="6.079064745s" totalImagesPullingTime="681.990797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:00.060136384 +0000 UTC m=+1288.089446615" watchObservedRunningTime="2026-10-30 01:03:00.079064745 +0000 UTC m=+1288.108374982"
Oct 30 01:03:00.699156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:00.699122    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:01.361146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:01.361113    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:02.647682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:02.647646    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:04.420102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.420068    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:04.788208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.788174    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"]
Oct 30 01:03:04.792303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.792279    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.840992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.840961    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"]
Oct 30 01:03:04.888391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888359    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888405    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888430    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frfl4\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888560    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888592    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888621    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888799    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888823    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.888964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.888947    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990343    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990396    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990438    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990600    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990626    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990724    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990831    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990894    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.990936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.990926    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfl4\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.991071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.991044    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.991168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.991131    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.991506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.991484    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.991604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.991502    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.991604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.991508    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.993045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.993019    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:04.993274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:04.993255    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:05.015292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.015268    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:05.015423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.015407    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frfl4\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4\") pod \"e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") " pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:05.039808 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.039733    4577 generic.go:356] "Generic (PLEG): container finished" podID="4bad413b-e468-498a-8343-29be621eeeae" containerID="8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d" exitCode=0
Oct 30 01:03:05.039962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.039819    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" event={"ID":"4bad413b-e468-498a-8343-29be621eeeae","Type":"ContainerDied","Data":"8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"}
Oct 30 01:03:05.040353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.040338    4577 scope.go:122] "RemoveContainer" containerID="8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"
Oct 30 01:03:05.042018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.041994    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:05.042077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.042061    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:05.042125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.042094    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:05.112583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.112555    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:05.495781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.495751    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"]
Oct 30 01:03:05.499990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.499957    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:05.500086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.500025    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:05.500086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:05.500052    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:06.010122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.010094    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:03:06.119064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.116407    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:06.119064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.116502    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:06.200638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.200597    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:06.200883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.200674    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" event={"ID":"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801","Type":"ContainerStarted","Data":"507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376"}
Oct 30 01:03:06.200883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.200711    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" event={"ID":"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801","Type":"ContainerStarted","Data":"744d98ab75fe7b0a574e6a8222ee023201a7813639378daa89ac0914b063ddd6"}
Oct 30 01:03:06.200883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.200741    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" event={"ID":"4bad413b-e468-498a-8343-29be621eeeae","Type":"ContainerStarted","Data":"e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f"}
Oct 30 01:03:06.205630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.205458    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podStartSLOduration=2.205438629 podStartE2EDuration="2.205438629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:06.194970371 +0000 UTC m=+1294.224280602" watchObservedRunningTime="2026-10-30 01:03:06.205438629 +0000 UTC m=+1294.234748861"
Oct 30 01:03:06.220646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.220621    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:06.273268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.273184    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:03:06.273628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.273602    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:06.273866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:06.273820    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:07.114323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.114284    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:07.114839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.114357    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:07.260967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.260230    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:07.260967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.260291    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:07.326673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.326634    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:07.335831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.335221    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:03:07.525908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:07.521999    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:08.133873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.131445    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:08.133873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.131543    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:08.194259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.194223    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:08.263031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.262999    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:08.263319 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.263290    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:08.819138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.819097    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:08.941068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:08.941032    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:09.113975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.113835    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:09.113975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.113926    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:09.115594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.115568    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"]
Oct 30 01:03:09.137296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.137267    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9b3887ff-mwf9f"]
Oct 30 01:03:09.258310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.258268    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:09.258612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.258579    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:09.719937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.719905    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:09.836222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:09.836182    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:10.241391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.240113    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:10.241391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.240170    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:10.273938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.263568    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:10.273938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.263640    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:10.648111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.639732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:10.761869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.745446    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4de41d5d-743f-4b07-8b08-b80fc05fdf5e" path="/var/lib/kubelet/pods/4de41d5d-743f-4b07-8b08-b80fc05fdf5e/volumes"
Oct 30 01:03:10.861908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.770982    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:10.920938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.914917    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"]
Oct 30 01:03:10.926250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:10.926219    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.010869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.005460    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-645126-openshift-default-dockercfg-29jmx\""
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.322953    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.323102    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.323165    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.323334    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.323526    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.323832    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.323952    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.338662    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.338741    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.348186    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddb9g\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.348346    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.349752    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:11.362990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.349834    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.514049    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.514385    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.525062    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.526720    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.526918    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527261    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527361    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527506    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddb9g\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527624    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527718    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527766    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.527903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.527806    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.555871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.555352    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.556062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.556018    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.556575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.556544    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.556828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.556804    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.557221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.557199    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.608869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.604498    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.625907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.610863    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"]
Oct 30 01:03:11.625907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.622414    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.641231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.641199    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.649899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.649585    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddb9g\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g\") pod \"e2e-api-a-645126-openshift-default-667d76b56f-q9bjj\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") " pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:11.810188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:11.810149    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:12.245957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.245899    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:12.295921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.270949    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:12.339796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.339755    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused" start-of-body=
Oct 30 01:03:12.340051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.339823    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.125:15021/healthz/ready\": dial tcp 10.134.0.125:15021: connect: connection refused"
Oct 30 01:03:12.514171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.510613    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:12.514171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.510702    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:12.514171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.510754    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:03:12.520370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.520345    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"]
Oct 30 01:03:12.633242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:12.633209    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:13.004066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:13.003924    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" event={"ID":"e737e0f8-63b5-48e0-8350-6791d948044a","Type":"ContainerStarted","Data":"dbc23c061ff89fd2a37df448bf5adfcf6ef8b536be466c213fc419cba8aaa587"}
Oct 30 01:03:13.113371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:13.113330    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 01:03:13.113562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:13.113406    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 01:03:13.338928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:13.338813    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:03:13.341262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:13.341241    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:03:14.015289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.015225    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" event={"ID":"e737e0f8-63b5-48e0-8350-6791d948044a","Type":"ContainerStarted","Data":"5f9b5941720bf406379ad7db75821e9c4273242c3f5e46a9f37a782a95e2af26"}
Oct 30 01:03:14.079538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.079482    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podStartSLOduration=4.079466101 podStartE2EDuration="4.079466101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:14.077242138 +0000 UTC m=+1302.106552397" watchObservedRunningTime="2026-10-30 01:03:14.079466101 +0000 UTC m=+1302.108776332"
Oct 30 01:03:14.121242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.121211    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:14.123631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.123606    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:14.124616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.124588    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:03:14.407822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.407733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:14.810932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.810903    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:14.816878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:14.816832    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:15.025277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:15.025248    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:15.026276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:15.026258    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:03:16.362033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:16.361998    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:16.537891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:16.534335    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:16.650266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:16.650179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:18.279394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:18.279361    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:18.410250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:18.410217    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:18.603927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:18.600477    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:19.000819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:19.000779    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:19.606218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:19.606183    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:20.116722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:20.116693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:20.244894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:20.238284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:20.511873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:20.511640    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:20.622563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:20.622525    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:21.387262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:21.387206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:21.511907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:21.511875    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:22.414125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:22.414033    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:23.123769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:23.123738    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"]
Oct 30 01:03:23.140050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:23.140018    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5c283c5e-9njvv"]
Oct 30 01:03:24.114823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.114790    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc1c6877-8cd5-4114-adfa-b9f69612aab9" path="/var/lib/kubelet/pods/fc1c6877-8cd5-4114-adfa-b9f69612aab9/volumes"
Oct 30 01:03:24.206012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.205981    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"]
Oct 30 01:03:24.212099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.212077    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.228668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.228638    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8363a40b-serving-cert\""
Oct 30 01:03:24.230580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.230556    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"]
Oct 30 01:03:24.307410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.307371    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:24.321278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.321245    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.321445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.321384    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.321491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.321444    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.321491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.321472    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswc9\" (UniqueName: \"kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.433931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.432955    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.433931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.433010    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswc9\" (UniqueName: \"kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.433931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.433347    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.433931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.433460    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.434422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.434350    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.479311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.478934    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:24.496971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.496925    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.498145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.498080    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.506005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.505529    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswc9\" (UniqueName: \"kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9\") pod \"maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.547218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.546954    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:24.635725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.635654    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:24.938182 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:03:24.938153    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2010e487_4018_4c27_897c_30ef38eae84a.slice/crio-9f1b74491b5eb564434ff279feba8556a9176d704d894eb3a140da1763b0d112 WatchSource:0}: Error finding container 9f1b74491b5eb564434ff279feba8556a9176d704d894eb3a140da1763b0d112: Status 404 returned error can't find the container with id 9f1b74491b5eb564434ff279feba8556a9176d704d894eb3a140da1763b0d112
Oct 30 01:03:24.940695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:24.940668    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"]
Oct 30 01:03:25.222976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:25.222935    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5" event={"ID":"2010e487-4018-4c27-897c-30ef38eae84a","Type":"ContainerStarted","Data":"9f1b74491b5eb564434ff279feba8556a9176d704d894eb3a140da1763b0d112"}
Oct 30 01:03:26.125362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:26.125261    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:26.233522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:26.233475    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5" event={"ID":"2010e487-4018-4c27-897c-30ef38eae84a","Type":"ContainerStarted","Data":"8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c"}
Oct 30 01:03:26.233965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:26.233586    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:26.266894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:26.264662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:26.285573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:26.285513    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5" podStartSLOduration=1.86000035 podStartE2EDuration="2.285493792s" totalImagesPullingTime="425.493442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:26.284363512 +0000 UTC m=+1314.313673762" watchObservedRunningTime="2026-10-30 01:03:26.285493792 +0000 UTC m=+1314.314804023"
Oct 30 01:03:27.023332 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:27.023296    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:03:28.244522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:28.242635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:30.007805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:30.007771    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:31.770885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:31.770812    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:32.248130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:32.248095    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:03:33.035782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:33.035752    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"]
Oct 30 01:03:33.049397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:33.049367    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-47047b78-bk59w"]
Oct 30 01:03:33.483049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:33.482990    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:33.593207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:33.593171    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:33.711909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:33.709421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:33.828858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:33.828752    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:34.112719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:34.112639    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af4c5fae-0415-46ea-bcdd-ba3713fc8ca4" path="/var/lib/kubelet/pods/af4c5fae-0415-46ea-bcdd-ba3713fc8ca4/volumes"
Oct 30 01:03:35.403537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:35.403490    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:37.325785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:37.325732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:38.779717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:38.779677    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"]
Oct 30 01:03:38.780270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:38.779918    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q" podUID="d5be7611-9bf8-46c0-881a-96611ebfb01d" containerName="maas-api" containerID="cri-o://f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e" gracePeriod=30
Oct 30 01:03:38.860535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:38.860499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:39.025789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.025680    4577 scope.go:122] "RemoveContainer" containerID="6d7aae0b57c7691a0ad855e49812ce39d5756e1134e5ff9d7632f8c6e9beb147"
Oct 30 01:03:39.043042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.043015    4577 scope.go:122] "RemoveContainer" containerID="713a7c70f0816b4905f2111c51b4f9b656a194ed32ebad68c43d3e97ab579b30"
Oct 30 01:03:39.099396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.099371    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:03:39.243403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.243376    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls\") pod \"d5be7611-9bf8-46c0-881a-96611ebfb01d\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") "
Oct 30 01:03:39.243575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.243467    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs\") pod \"d5be7611-9bf8-46c0-881a-96611ebfb01d\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") "
Oct 30 01:03:39.243619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.243581    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz\" (UniqueName: \"kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz\") pod \"d5be7611-9bf8-46c0-881a-96611ebfb01d\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") "
Oct 30 01:03:39.243680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.243664    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca\") pod \"d5be7611-9bf8-46c0-881a-96611ebfb01d\" (UID: \"d5be7611-9bf8-46c0-881a-96611ebfb01d\") "
Oct 30 01:03:39.244253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.244095    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca" pod "d5be7611-9bf8-46c0-881a-96611ebfb01d" (UID: "d5be7611-9bf8-46c0-881a-96611ebfb01d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:39.244610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.244589    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5be7611-9bf8-46c0-881a-96611ebfb01d-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:39.245835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.245810    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls" pod "d5be7611-9bf8-46c0-881a-96611ebfb01d" (UID: "d5be7611-9bf8-46c0-881a-96611ebfb01d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:39.246062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.246042    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz" pod "d5be7611-9bf8-46c0-881a-96611ebfb01d" (UID: "d5be7611-9bf8-46c0-881a-96611ebfb01d"). InnerVolumeSpecName "kube-api-access-7d7kz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:39.246128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.246087    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs" pod "d5be7611-9bf8-46c0-881a-96611ebfb01d" (UID: "d5be7611-9bf8-46c0-881a-96611ebfb01d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:39.358934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.347241    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:39.358934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.347274    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5be7611-9bf8-46c0-881a-96611ebfb01d-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:39.358934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.347288    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d7kz\" (UniqueName: \"kubernetes.io/projected/d5be7611-9bf8-46c0-881a-96611ebfb01d-kube-api-access-7d7kz\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:39.389518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.389479    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:39.445918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.420665    4577 generic.go:356] "Generic (PLEG): container finished" podID="d5be7611-9bf8-46c0-881a-96611ebfb01d" containerID="f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e" exitCode=0
Oct 30 01:03:39.445918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.420871    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q" event={"ID":"d5be7611-9bf8-46c0-881a-96611ebfb01d","Type":"ContainerDied","Data":"f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e"}
Oct 30 01:03:39.445918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.420930    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q" event={"ID":"d5be7611-9bf8-46c0-881a-96611ebfb01d","Type":"ContainerDied","Data":"ca28c190b1eaa57cf76693d9a387e6c0d8ace1d67847383c3e9bb9df2b51ff87"}
Oct 30 01:03:39.445918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.420966    4577 scope.go:122] "RemoveContainer" containerID="f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e"
Oct 30 01:03:39.445918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.421150    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"
Oct 30 01:03:39.488888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.486029    4577 scope.go:122] "RemoveContainer" containerID="f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e"
Oct 30 01:03:39.488888 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:39.486532    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e\": container with ID starting with f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e not found: ID does not exist" containerID="f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e"
Oct 30 01:03:39.488888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.486575    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e"} err="failed to get container status \"f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e\": rpc error: code = NotFound desc = could not find container \"f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e\": container with ID starting with f202a573bce2e96fd365c4dae1c0cc9e59d8c75b7da175e392159613fd713e8e not found: ID does not exist"
Oct 30 01:03:39.515888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.511080    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:39.576870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.568898    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"]
Oct 30 01:03:39.631936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.628006    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:39.632704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:39.632670    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-153746-64b89749fb-zxn5q"]
Oct 30 01:03:40.030291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:40.030261    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:40.113335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:40.113304    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5be7611-9bf8-46c0-881a-96611ebfb01d" path="/var/lib/kubelet/pods/d5be7611-9bf8-46c0-881a-96611ebfb01d/volumes"
Oct 30 01:03:41.110210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.110175    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"]
Oct 30 01:03:41.110683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.110432    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" podUID="83b82600-a7c0-4ab2-bce1-e36c3709e16b" containerName="maas-api" containerID="cri-o://1ac822b5682a9492b21b6d52258c7e7d523cfedc6f1cf3d654fe3338113f9a51" gracePeriod=30
Oct 30 01:03:41.171328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.171290    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:41.644876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.644804    4577 generic.go:356] "Generic (PLEG): container finished" podID="83b82600-a7c0-4ab2-bce1-e36c3709e16b" containerID="1ac822b5682a9492b21b6d52258c7e7d523cfedc6f1cf3d654fe3338113f9a51" exitCode=0
Oct 30 01:03:41.645330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.645206    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" event={"ID":"83b82600-a7c0-4ab2-bce1-e36c3709e16b","Type":"ContainerDied","Data":"1ac822b5682a9492b21b6d52258c7e7d523cfedc6f1cf3d654fe3338113f9a51"}
Oct 30 01:03:41.658945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.654044    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:03:41.780228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.780198    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:41.784661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.784631    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg\" (UniqueName: \"kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg\") pod \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") "
Oct 30 01:03:41.784832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.784809    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca\") pod \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") "
Oct 30 01:03:41.784931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.784900    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\") pod \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") "
Oct 30 01:03:41.785047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.785031    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs\") pod \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\" (UID: \"83b82600-a7c0-4ab2-bce1-e36c3709e16b\") "
Oct 30 01:03:41.785349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.785304    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca" pod "83b82600-a7c0-4ab2-bce1-e36c3709e16b" (UID: "83b82600-a7c0-4ab2-bce1-e36c3709e16b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:41.785805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.785781    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83b82600-a7c0-4ab2-bce1-e36c3709e16b-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:41.788139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.788118    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg" pod "83b82600-a7c0-4ab2-bce1-e36c3709e16b" (UID: "83b82600-a7c0-4ab2-bce1-e36c3709e16b"). InnerVolumeSpecName "kube-api-access-c6xbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:41.788246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.788198    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs" pod "83b82600-a7c0-4ab2-bce1-e36c3709e16b" (UID: "83b82600-a7c0-4ab2-bce1-e36c3709e16b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:41.788246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.788207    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls" pod "83b82600-a7c0-4ab2-bce1-e36c3709e16b" (UID: "83b82600-a7c0-4ab2-bce1-e36c3709e16b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:41.919789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.912301    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:41.919789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.912337    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83b82600-a7c0-4ab2-bce1-e36c3709e16b-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:41.919789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.912354    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6xbg\" (UniqueName: \"kubernetes.io/projected/83b82600-a7c0-4ab2-bce1-e36c3709e16b-kube-api-access-c6xbg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:41.930675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:41.924515    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:42.042547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.042450    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:42.438705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.438648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:42.657420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.657330    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"
Oct 30 01:03:42.657420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.657347    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4" event={"ID":"83b82600-a7c0-4ab2-bce1-e36c3709e16b","Type":"ContainerDied","Data":"c32977c9b5a26d19a14fd8e35489966b7c4055218e7a05925b5aad2f0864e49d"}
Oct 30 01:03:42.657636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.657431    4577 scope.go:122] "RemoveContainer" containerID="1ac822b5682a9492b21b6d52258c7e7d523cfedc6f1cf3d654fe3338113f9a51"
Oct 30 01:03:42.718375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.718341    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"]
Oct 30 01:03:42.731614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:42.731577    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6482ae-569f5d8b5f-8ppm4"]
Oct 30 01:03:43.656168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656091    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"]
Oct 30 01:03:43.656798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656779    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d5be7611-9bf8-46c0-881a-96611ebfb01d" containerName="maas-api"
Oct 30 01:03:43.656798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656798    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5be7611-9bf8-46c0-881a-96611ebfb01d" containerName="maas-api"
Oct 30 01:03:43.656966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656822    4577 cpu_manager.go:414] "containerMap: removing container" podUID="83b82600-a7c0-4ab2-bce1-e36c3709e16b" containerName="maas-api"
Oct 30 01:03:43.656966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656827    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83b82600-a7c0-4ab2-bce1-e36c3709e16b" containerName="maas-api"
Oct 30 01:03:43.656966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656956    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5be7611-9bf8-46c0-881a-96611ebfb01d" containerName="maas-api"
Oct 30 01:03:43.657099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.656968    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="83b82600-a7c0-4ab2-bce1-e36c3709e16b" containerName="maas-api"
Oct 30 01:03:43.662133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.662114    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.686043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.686017    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"]
Oct 30 01:03:43.691897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.691824    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-1c78b8-serving-cert\""
Oct 30 01:03:43.723451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.723410    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.723623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.723518    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.723623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.723545    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.723623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.723584    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl7hf\" (UniqueName: \"kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.743780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.743750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:43.825060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.825021    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.825262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.825113    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.825262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.825151    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.825262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.825186    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl7hf\" (UniqueName: \"kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.825948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.825925    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.827747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.827716    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.827839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.827802    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.853411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.853373    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:43.854121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.854097    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl7hf\" (UniqueName: \"kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf\") pod \"maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:43.995510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:43.995468    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:44.009601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:44.009568    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:44.132046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:44.132010    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83b82600-a7c0-4ab2-bce1-e36c3709e16b" path="/var/lib/kubelet/pods/83b82600-a7c0-4ab2-bce1-e36c3709e16b/volumes"
Oct 30 01:03:44.417148 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:03:44.417118    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ed7daf_92a2_4bc9_a548_fbdac75692a2.slice/crio-5dc4a63e1a93584f2a1394a5aec26017c7c38767c4d068bc3a564bd39c377ca9 WatchSource:0}: Error finding container 5dc4a63e1a93584f2a1394a5aec26017c7c38767c4d068bc3a564bd39c377ca9: Status 404 returned error can't find the container with id 5dc4a63e1a93584f2a1394a5aec26017c7c38767c4d068bc3a564bd39c377ca9
Oct 30 01:03:44.417963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:44.417940    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"]
Oct 30 01:03:44.684578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:44.684500    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff" event={"ID":"b3ed7daf-92a2-4bc9-a548-fbdac75692a2","Type":"ContainerStarted","Data":"5dc4a63e1a93584f2a1394a5aec26017c7c38767c4d068bc3a564bd39c377ca9"}
Oct 30 01:03:45.624566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:45.624522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:45.781918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:45.781877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff" event={"ID":"b3ed7daf-92a2-4bc9-a548-fbdac75692a2","Type":"ContainerStarted","Data":"6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183"}
Oct 30 01:03:45.793920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:45.793887    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:45.906919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:45.885887    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff" podStartSLOduration=2.496829517 podStartE2EDuration="2.88586671s" totalImagesPullingTime="389.037193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:45.844275704 +0000 UTC m=+1333.873585938" watchObservedRunningTime="2026-10-30 01:03:45.88586671 +0000 UTC m=+1333.915176939"
Oct 30 01:03:46.847741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:46.847702    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:46.966433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:46.966394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:47.204041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:47.203958    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"]
Oct 30 01:03:47.204258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:47.204226    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" containerID="cri-o://325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759" gracePeriod=30
Oct 30 01:03:47.262121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:47.262088    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:47.514473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:47.514435    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:48.040684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:48.040653    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"]
Oct 30 01:03:48.055369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:48.055335    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fd554126-p8dnc"]
Oct 30 01:03:48.115039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:48.115002    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f38c861-1a2a-48a8-85dc-698fc63c8897" path="/var/lib/kubelet/pods/8f38c861-1a2a-48a8-85dc-698fc63c8897/volumes"
Oct 30 01:03:48.840303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:48.840265    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:48.961349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:48.961313    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:49.181690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:49.181595    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"]
Oct 30 01:03:49.182506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:49.182470    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" containerID="cri-o://c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541" gracePeriod=30
Oct 30 01:03:49.221828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:49.221794    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:49.285547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:49.285496    4577 patch_prober.go:29] interesting pod/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:03:49.285791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:49.285570    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:03:49.341718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:49.341674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:51.205244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:51.205209    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:52.526752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.526726    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:03:52.664262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664232    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664276    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664323    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664383    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664468    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664504    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664548    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664554    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.664768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664594    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.664768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664647    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs\") pod \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\" (UID: \"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0\") "
Oct 30 01:03:52.665020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.664892    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:52.665081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665043    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.665198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665177    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.665333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665315    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.665708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665683    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.665813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665708    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.665813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665722    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.665813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665735    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.665813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.665749    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.669460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.669433    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.669570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.669468    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:03:52.669570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.669489    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "kube-api-access-msrz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:52.669935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.669827    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token" pod "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" (UID: "18569ae2-d2f0-4236-bf8c-9d0a5096d7c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:52.699908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.699877    4577 patch_prober.go:29] interesting pod/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:03:52.700051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.699951    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:03:52.767697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.767595    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.767697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.767633    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.767697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.767648    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.767697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.767660    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msrz7\" (UniqueName: \"kubernetes.io/projected/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0-kube-api-access-msrz7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.825029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.825001    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:03:52.866268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.866233    4577 generic.go:356] "Generic (PLEG): container finished" podID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerID="325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759" exitCode=0
Oct 30 01:03:52.866470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.866308    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"
Oct 30 01:03:52.866470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.866311    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" event={"ID":"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0","Type":"ContainerDied","Data":"325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759"}
Oct 30 01:03:52.866470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.866375    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77" event={"ID":"18569ae2-d2f0-4236-bf8c-9d0a5096d7c0","Type":"ContainerDied","Data":"df1f0897503736714b028fc27d6cfe68f927ffb8180831b7899cef455cb594eb"}
Oct 30 01:03:52.866470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.866398    4577 scope.go:122] "RemoveContainer" containerID="325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759"
Oct 30 01:03:52.895002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.894884    4577 scope.go:122] "RemoveContainer" containerID="046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"
Oct 30 01:03:52.909030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.909002    4577 scope.go:122] "RemoveContainer" containerID="325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759"
Oct 30 01:03:52.909456 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:52.909433    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759\": container with ID starting with 325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759 not found: ID does not exist" containerID="325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759"
Oct 30 01:03:52.909558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.909463    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759"} err="failed to get container status \"325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759\": rpc error: code = NotFound desc = could not find container \"325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759\": container with ID starting with 325a61fae3ff5aeb03f98b7b235d57f403628490c391a2d8e0b2bd34bed97759 not found: ID does not exist"
Oct 30 01:03:52.909558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.909483    4577 scope.go:122] "RemoveContainer" containerID="046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"
Oct 30 01:03:52.909808 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:52.909789    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100\": container with ID starting with 046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100 not found: ID does not exist" containerID="046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"
Oct 30 01:03:52.909912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.909815    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100"} err="failed to get container status \"046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100\": rpc error: code = NotFound desc = could not find container \"046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100\": container with ID starting with 046043772dac4c94e7caf073562de325d7a7ae657ad43f7f8a22cf7c63be1100 not found: ID does not exist"
Oct 30 01:03:52.940785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.940752    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"]
Oct 30 01:03:52.948906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:52.947215    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-153746-openshift-default-5bbcb855f9-h5t77"]
Oct 30 01:03:53.237985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:53.237958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:54.113536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.113503    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" path="/var/lib/kubelet/pods/18569ae2-d2f0-4236-bf8c-9d0a5096d7c0/volumes"
Oct 30 01:03:54.479332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.479307    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:03:54.515515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515482    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.515666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515579    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.515666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515600    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.515766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515698    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.515766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515713    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:54.515928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515775    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.515928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515816    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.515928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515900    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.516086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515935    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.516086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.515971    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo\") pod \"5f849b4b-9407-4325-bb99-6fac3dcc042f\" (UID: \"5f849b4b-9407-4325-bb99-6fac3dcc042f\") "
Oct 30 01:03:54.516086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516073    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:54.516259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516154    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:54.516259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516206    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:54.516466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516440    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:54.516999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516906    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.516999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516932    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f849b4b-9407-4325-bb99-6fac3dcc042f-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.516999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516946    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.516999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516961    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.516999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.516974    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.518198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.518175    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:54.518298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.518219    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:54.518360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.518328    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "kube-api-access-9trnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:54.518540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.518525    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo" pod "5f849b4b-9407-4325-bb99-6fac3dcc042f" (UID: "5f849b4b-9407-4325-bb99-6fac3dcc042f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:03:54.619024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.618986    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9trnj\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-kube-api-access-9trnj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.619024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.619014    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.619024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.619023    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.619024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.619032    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f849b4b-9407-4325-bb99-6fac3dcc042f-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.897251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.897212    4577 generic.go:356] "Generic (PLEG): container finished" podID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerID="c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541" exitCode=0
Oct 30 01:03:54.897548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.897268    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"
Oct 30 01:03:54.897548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.897290    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" event={"ID":"5f849b4b-9407-4325-bb99-6fac3dcc042f","Type":"ContainerDied","Data":"c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541"}
Oct 30 01:03:54.897548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.897351    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp" event={"ID":"5f849b4b-9407-4325-bb99-6fac3dcc042f","Type":"ContainerDied","Data":"797025e2d412975f85f8ca908c2c144a735be8c9644c20a1c7c76afe6f6e0d68"}
Oct 30 01:03:54.897548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.897390    4577 scope.go:122] "RemoveContainer" containerID="c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541"
Oct 30 01:03:54.909919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.909897    4577 scope.go:122] "RemoveContainer" containerID="ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"
Oct 30 01:03:54.920321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.920300    4577 scope.go:122] "RemoveContainer" containerID="c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541"
Oct 30 01:03:54.920615 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:54.920595    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541\": container with ID starting with c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541 not found: ID does not exist" containerID="c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541"
Oct 30 01:03:54.920670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.920624    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541"} err="failed to get container status \"c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541\": rpc error: code = NotFound desc = could not find container \"c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541\": container with ID starting with c0d5e7e630294a4c23e0c4793d9c9c3340788ee71d22b3289c54fa3a731f8541 not found: ID does not exist"
Oct 30 01:03:54.920670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.920642    4577 scope.go:122] "RemoveContainer" containerID="ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"
Oct 30 01:03:54.920908 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:54.920888    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab\": container with ID starting with ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab not found: ID does not exist" containerID="ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"
Oct 30 01:03:54.920971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.920918    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab"} err="failed to get container status \"ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab\": rpc error: code = NotFound desc = could not find container \"ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab\": container with ID starting with ef7c262c4595467601f4dc9c68093a2289adcc4dac004c6e7669c190df2b3aab not found: ID does not exist"
Oct 30 01:03:54.958383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.958350    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"]
Oct 30 01:03:54.971384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:54.971344    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6482ae-openshift-default-bd4ccc9cc-xmztp"]
Oct 30 01:03:55.020437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:55.020401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:56.033742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:56.033714    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"]
Oct 30 01:03:56.047063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:56.047034    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e90116f8-6zq49"]
Oct 30 01:03:56.113075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:56.113047    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fc9ef92-672b-4f84-b0fa-0b15a86bcf20" path="/var/lib/kubelet/pods/2fc9ef92-672b-4f84-b0fa-0b15a86bcf20/volumes"
Oct 30 01:03:56.113605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:56.113589    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" path="/var/lib/kubelet/pods/5f849b4b-9407-4325-bb99-6fac3dcc042f/volumes"
Oct 30 01:03:56.772771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:56.771761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:03:57.065652 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:03:57.065529    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:03:58.701321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:03:58.701287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:00.093869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:00.086907    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"]
Oct 30 01:04:00.218518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:00.218456    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-557d1ba7-s7wt6"]
Oct 30 01:04:00.427916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:00.422664    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:00.690944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:00.656032    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:00.834867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:00.815600    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:02.114425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:02.114389    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d536388-1129-4593-b786-e18909331475" path="/var/lib/kubelet/pods/5d536388-1129-4593-b786-e18909331475/volumes"
Oct 30 01:04:02.276149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:02.276114    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:02.719224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:02.719187    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:03.107832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.107794    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.469341    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"]
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470509    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470531    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470568    4577 cpu_manager.go:414] "containerMap: removing container" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470579    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470596    4577 cpu_manager.go:414] "containerMap: removing container" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470604    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470790    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470808    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="18569ae2-d2f0-4236-bf8c-9d0a5096d7c0" containerName="istio-proxy"
Oct 30 01:04:03.482922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.470824    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy"
Oct 30 01:04:03.507475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.503736    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"]
Oct 30 01:04:03.507475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.503956    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.523284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.520119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:03.528183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.528161    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-76f654-openshift-default-dockercfg-p6m9l\""
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634182    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634252    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634329    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634377    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634434    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634558    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634632    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634679    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r9kw\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.726922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.634797    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.793957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.787763    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.855760    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.855869    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.855917    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r9kw\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856018    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856120    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856165    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856235    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856282    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856338    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.856922    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.857178    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.857733    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.862070    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:03.868872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:03.862376    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:04.071581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.071495    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r9kw\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:04.072516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.072460    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:04.100807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.100721    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:04.106172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.106143    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:04.128110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.128079    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:04.598785 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:04:04.598751    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod765e057d_2b90_442f_bec5_78841a2d9619.slice/crio-841fc51a2ae4f9e94a3c6cf9b8e15724a4142c9dba5e81edf1f4fedfdcaf742c WatchSource:0}: Error finding container 841fc51a2ae4f9e94a3c6cf9b8e15724a4142c9dba5e81edf1f4fedfdcaf742c: Status 404 returned error can't find the container with id 841fc51a2ae4f9e94a3c6cf9b8e15724a4142c9dba5e81edf1f4fedfdcaf742c
Oct 30 01:04:04.599910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.599886    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"]
Oct 30 01:04:04.601829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.601799    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:04.601946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.601886    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:04.601946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.601920    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:04.608403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:04.608380    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:05.243643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:05.243614    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" event={"ID":"765e057d-2b90-442f-bec5-78841a2d9619","Type":"ContainerStarted","Data":"421e190c7c7910cb2d8389f448893e5260779a1c21b9826fcea0f3d883a93b4d"}
Oct 30 01:04:05.243836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:05.243661    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" event={"ID":"765e057d-2b90-442f-bec5-78841a2d9619","Type":"ContainerStarted","Data":"841fc51a2ae4f9e94a3c6cf9b8e15724a4142c9dba5e81edf1f4fedfdcaf742c"}
Oct 30 01:04:05.326002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:05.325953    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podStartSLOduration=2.325938492 podStartE2EDuration="2.325938492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:05.311191475 +0000 UTC m=+1353.340501723" watchObservedRunningTime="2026-10-30 01:04:05.325938492 +0000 UTC m=+1353.355248722"
Oct 30 01:04:06.031361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.031325    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"]
Oct 30 01:04:06.066653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.049536    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ba393b1f-l9t7r"]
Oct 30 01:04:06.113323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.113290    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf" path="/var/lib/kubelet/pods/5a26825c-3de9-4ff9-a705-5c0b4f4ad4bf/volumes"
Oct 30 01:04:06.129390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.129356    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:06.135151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.135131    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:06.329921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.329420    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:06.338525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.338206    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:04:06.527958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:06.519170    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:08.089358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:08.089268    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:08.239312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:08.239259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:08.352081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:08.352045    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:10.039498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:10.039459    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"]
Oct 30 01:04:10.053481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:10.053438    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-767c4416-65mwc"]
Oct 30 01:04:10.086946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:10.086911    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:10.126885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:10.126826    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3887e40d-c73d-4337-aab8-3b76cfb90bad" path="/var/lib/kubelet/pods/3887e40d-c73d-4337-aab8-3b76cfb90bad/volumes"
Oct 30 01:04:10.206104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:10.204704    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:11.937904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:11.937835    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:13.381997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.381965    4577 generic.go:356] "Generic (PLEG): container finished" podID="e737e0f8-63b5-48e0-8350-6791d948044a" containerID="5f9b5941720bf406379ad7db75821e9c4273242c3f5e46a9f37a782a95e2af26" exitCode=0
Oct 30 01:04:13.382539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.382058    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" event={"ID":"e737e0f8-63b5-48e0-8350-6791d948044a","Type":"ContainerDied","Data":"5f9b5941720bf406379ad7db75821e9c4273242c3f5e46a9f37a782a95e2af26"}
Oct 30 01:04:13.382660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.382640    4577 scope.go:122] "RemoveContainer" containerID="5f9b5941720bf406379ad7db75821e9c4273242c3f5e46a9f37a782a95e2af26"
Oct 30 01:04:13.384726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.384692    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:13.384837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.384789    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:13.384921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.384835    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:13.709866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:13.709767    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:14.032473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.032436    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"]
Oct 30 01:04:14.036941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.036909    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4972fecd-7k9qc"]
Oct 30 01:04:14.113890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.113813    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1b9f812-5e0e-4998-82eb-6b97a7a155a2" path="/var/lib/kubelet/pods/a1b9f812-5e0e-4998-82eb-6b97a7a155a2/volumes"
Oct 30 01:04:14.392832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.392740    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" event={"ID":"e737e0f8-63b5-48e0-8350-6791d948044a","Type":"ContainerStarted","Data":"9ee6b03ce406f3186f5e2bef2c886afb9b74fbfd7560e1d26953bc1aaa36a6d4"}
Oct 30 01:04:14.810516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.810479    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:04:14.811627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.811602    4577 patch_prober.go:29] interesting pod/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 30 01:04:14.811769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:14.811664    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 30 01:04:15.025068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:15.025037    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:04:15.499219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:15.499186    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:15.810626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:15.810545    4577 patch_prober.go:29] interesting pod/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 30 01:04:15.810626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:15.810602    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 30 01:04:16.810635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:16.810602    4577 patch_prober.go:29] interesting pod/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 30 01:04:16.811072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:16.810668    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 30 01:04:17.387757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:17.387721    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:17.722877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:17.722778    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:17.810635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:17.810590    4577 patch_prober.go:29] interesting pod/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 30 01:04:17.811101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:17.810664    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 30 01:04:17.883428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:17.883388    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:18.810832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:18.810791    4577 patch_prober.go:29] interesting pod/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 30 01:04:18.811263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:18.810872    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 30 01:04:19.320254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.320215    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:19.435255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.435220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:19.562499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.562455    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:19.578964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.578869    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"]
Oct 30 01:04:19.579721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.579698    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy"
Oct 30 01:04:19.579721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.579720    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy"
Oct 30 01:04:19.579997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.579980    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f849b4b-9407-4325-bb99-6fac3dcc042f" containerName="istio-proxy"
Oct 30 01:04:19.582835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.582815    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:19.703791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.703759    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-76f654-dockercfg-q7vnn\""
Oct 30 01:04:19.704165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.704144    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-76f654\""
Oct 30 01:04:19.824872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.772445    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"]
Oct 30 01:04:19.824872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.814585    4577 patch_prober.go:29] interesting pod/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 30 01:04:19.824872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.814641    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 30 01:04:19.824872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.824048    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26j9q\" (UniqueName: \"kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q\") pod \"payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:19.824872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:19.824195    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:20.003695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.002019    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26j9q\" (UniqueName: \"kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q\") pod \"payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:20.003695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.002174    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:20.003695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.003352    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:20.006934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.004115    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"]
Oct 30 01:04:20.010065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.010037    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.015282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.015208    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"]
Oct 30 01:04:20.061830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.061795    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26j9q\" (UniqueName: \"kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q\") pod \"payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:20.113823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.113793    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") " pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.114573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.114546    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjms8\" (UniqueName: \"kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8\") pod \"payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") " pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.197536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.197505    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:20.217760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.217722    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjms8\" (UniqueName: \"kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8\") pod \"payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") " pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.218041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.218017    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") " pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.218575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.218550    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") " pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.243154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.243129    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjms8\" (UniqueName: \"kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8\") pod \"payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") " pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.366917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.366886    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:20.512330 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:04:20.512300    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063f74ab_081d_4447_bcf2_f3cf744d7b1c.slice/crio-faf5f498c67bf76a77fbfb0c22f082b616ed760f4b363619f584d18973b6ed28 WatchSource:0}: Error finding container faf5f498c67bf76a77fbfb0c22f082b616ed760f4b363619f584d18973b6ed28: Status 404 returned error can't find the container with id faf5f498c67bf76a77fbfb0c22f082b616ed760f4b363619f584d18973b6ed28
Oct 30 01:04:20.512613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.512593    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"]
Oct 30 01:04:20.673130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.673099    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"]
Oct 30 01:04:20.673532 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:04:20.673511    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f3c030_7ed2_4b0e_b9e6_0f38cac84ef5.slice/crio-ca5baae5fd707dc54c0c0ac5daa7772983c844b0692f64a86736644152e23d98 WatchSource:0}: Error finding container ca5baae5fd707dc54c0c0ac5daa7772983c844b0692f64a86736644152e23d98: Status 404 returned error can't find the container with id ca5baae5fd707dc54c0c0ac5daa7772983c844b0692f64a86736644152e23d98
Oct 30 01:04:20.685212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.685171    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:20.816628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.816598    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:04:20.817872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:20.817837    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:04:21.193711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.193576    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:21.461249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.461156    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb" event={"ID":"063f74ab-081d-4447-bcf2-f3cf744d7b1c","Type":"ContainerStarted","Data":"95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778"}
Oct 30 01:04:21.461249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.461205    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb" event={"ID":"063f74ab-081d-4447-bcf2-f3cf744d7b1c","Type":"ContainerStarted","Data":"faf5f498c67bf76a77fbfb0c22f082b616ed760f4b363619f584d18973b6ed28"}
Oct 30 01:04:21.461249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.461247    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:21.463227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.463193    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr" event={"ID":"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5","Type":"ContainerStarted","Data":"7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12"}
Oct 30 01:04:21.463383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.463253    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr" event={"ID":"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5","Type":"ContainerStarted","Data":"ca5baae5fd707dc54c0c0ac5daa7772983c844b0692f64a86736644152e23d98"}
Oct 30 01:04:21.463383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.463281    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:21.511928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.511829    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb" podStartSLOduration=2.511810397 podStartE2EDuration="2.511810397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:21.506617521 +0000 UTC m=+1369.535927754" watchObservedRunningTime="2026-10-30 01:04:21.511810397 +0000 UTC m=+1369.541120629"
Oct 30 01:04:21.554255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.554202    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr" podStartSLOduration=2.554183573 podStartE2EDuration="2.554183573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:21.551117536 +0000 UTC m=+1369.580427768" watchObservedRunningTime="2026-10-30 01:04:21.554183573 +0000 UTC m=+1369.583493804"
Oct 30 01:04:21.639324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.639286    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:21.770419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:21.770386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:23.011667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:23.011635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:24.030810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.030778    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"]
Oct 30 01:04:24.044150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.044127    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c93eea16-f2xh7"]
Oct 30 01:04:24.114548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.114506    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21b5a82e-4596-49f2-b487-a402602408d3" path="/var/lib/kubelet/pods/21b5a82e-4596-49f2-b487-a402602408d3/volumes"
Oct 30 01:04:24.397605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.397528    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"]
Oct 30 01:04:24.397782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.397727    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5" podUID="2010e487-4018-4c27-897c-30ef38eae84a" containerName="maas-api" containerID="cri-o://8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c" gracePeriod=30
Oct 30 01:04:24.473169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.473132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:24.877874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.875058    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:04:24.969226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:24.969144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:25.077454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.077362    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls\") pod \"2010e487-4018-4c27-897c-30ef38eae84a\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") "
Oct 30 01:04:25.077968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.077485    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs\") pod \"2010e487-4018-4c27-897c-30ef38eae84a\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") "
Oct 30 01:04:25.077968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.077591    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca\") pod \"2010e487-4018-4c27-897c-30ef38eae84a\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") "
Oct 30 01:04:25.077968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.077627    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9\" (UniqueName: \"kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9\") pod \"2010e487-4018-4c27-897c-30ef38eae84a\" (UID: \"2010e487-4018-4c27-897c-30ef38eae84a\") "
Oct 30 01:04:25.124024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.123976    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca" pod "2010e487-4018-4c27-897c-30ef38eae84a" (UID: "2010e487-4018-4c27-897c-30ef38eae84a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:25.215866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.212572    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:25.241899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.237712    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2010e487-4018-4c27-897c-30ef38eae84a-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:25.241899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.241005    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs" pod "2010e487-4018-4c27-897c-30ef38eae84a" (UID: "2010e487-4018-4c27-897c-30ef38eae84a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:25.260421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.257444    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls" pod "2010e487-4018-4c27-897c-30ef38eae84a" (UID: "2010e487-4018-4c27-897c-30ef38eae84a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:25.304104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.301910    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9" pod "2010e487-4018-4c27-897c-30ef38eae84a" (UID: "2010e487-4018-4c27-897c-30ef38eae84a"). InnerVolumeSpecName "kube-api-access-vswc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:25.345919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.342590    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:25.345919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.342626    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2010e487-4018-4c27-897c-30ef38eae84a-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:25.345919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.342641    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vswc9\" (UniqueName: \"kubernetes.io/projected/2010e487-4018-4c27-897c-30ef38eae84a-kube-api-access-vswc9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:25.475877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.471084    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:25.529099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.529045    4577 generic.go:356] "Generic (PLEG): container finished" podID="2010e487-4018-4c27-897c-30ef38eae84a" containerID="8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c" exitCode=0
Oct 30 01:04:25.529274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.529186    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5" event={"ID":"2010e487-4018-4c27-897c-30ef38eae84a","Type":"ContainerDied","Data":"8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c"}
Oct 30 01:04:25.529274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.529237    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5" event={"ID":"2010e487-4018-4c27-897c-30ef38eae84a","Type":"ContainerDied","Data":"9f1b74491b5eb564434ff279feba8556a9176d704d894eb3a140da1763b0d112"}
Oct 30 01:04:25.529274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.529265    4577 scope.go:122] "RemoveContainer" containerID="8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c"
Oct 30 01:04:25.529478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.529463    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"
Oct 30 01:04:25.576479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.576453    4577 scope.go:122] "RemoveContainer" containerID="8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c"
Oct 30 01:04:25.576886 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:25.576829    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c\": container with ID starting with 8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c not found: ID does not exist" containerID="8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c"
Oct 30 01:04:25.576992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.576896    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c"} err="failed to get container status \"8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c\": rpc error: code = NotFound desc = could not find container \"8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c\": container with ID starting with 8f7ee3d6d56c78fa8b7899269e74abb132bfb0720d9259a4fae7a1075e7adb7c not found: ID does not exist"
Oct 30 01:04:25.612596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.610282    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"]
Oct 30 01:04:25.630902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.626009    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8363a40b-5dbd55b679-sjkp5"]
Oct 30 01:04:25.739229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:25.739192    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:26.112902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:26.112837    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2010e487-4018-4c27-897c-30ef38eae84a" path="/var/lib/kubelet/pods/2010e487-4018-4c27-897c-30ef38eae84a/volumes"
Oct 30 01:04:27.015177 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:27.015137    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:04:27.100765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:27.100729    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:29.775809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.775778    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:29.866380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.862952    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"]
Oct 30 01:04:29.866380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.864082    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2010e487-4018-4c27-897c-30ef38eae84a" containerName="maas-api"
Oct 30 01:04:29.866380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.864100    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2010e487-4018-4c27-897c-30ef38eae84a" containerName="maas-api"
Oct 30 01:04:29.866380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.864276    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2010e487-4018-4c27-897c-30ef38eae84a" containerName="maas-api"
Oct 30 01:04:29.879656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.874177    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.887049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.886995    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"]
Oct 30 01:04:29.921534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.919788    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-d4622a-openshift-default-dockercfg-rrvfd\""
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969124    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969183    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969261    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6skp\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969294    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969374    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969413    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969499    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969537    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:29.970900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.969577    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.014888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:29.998623    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:30.101873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.074961    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.119986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.119947    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.139151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.139112    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6skp\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.139436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.139415    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.166870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.139617    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.198001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.197959    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.198373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.198349    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.198561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.198541    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.198731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.198714    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.330395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.208522    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.330395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.086479    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.330395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.159357    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.330395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.168410    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.330395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.167773    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.510869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.504127    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.596975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.514158    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:04:30.596975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.521780    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.596975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.537481    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.596975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.561390    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6skp\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp\") pod \"e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") " pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:30.596975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.563294    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:30.748461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.748417    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:30.815891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:30.815720    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:31.261541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:31.261508    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"]
Oct 30 01:04:31.324920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:31.295003    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3c0179e3-725fk"]
Oct 30 01:04:32.025872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.018470    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:32.287900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.285200    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/health\": context deadline exceeded"
Oct 30 01:04:32.552559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.552509    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:32.632992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.632937    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:32.633455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.633422    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:32.791393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.791352    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e" path="/var/lib/kubelet/pods/a1b2d8b5-fcd8-49b5-817f-7bdf2f32df6e/volumes"
Oct 30 01:04:32.792208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.792183    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"]
Oct 30 01:04:32.792492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.792476    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:04:32.792699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.792685    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:04:32.820200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:32.820093    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:04:33.395688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:33.395569    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:33.787096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:33.787050    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" event={"ID":"421db701-5ef1-4a24-8fc6-342e88a351bf","Type":"ContainerStarted","Data":"8faaefd7f0a3229c6e0c25835c3978bf9eafda7f8467db0a9ac69dc1c43e111e"}
Oct 30 01:04:33.834568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:33.834520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:33.914138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:33.914110    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"]
Oct 30 01:04:33.946036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:33.944693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:34.031952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.027014    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"]
Oct 30 01:04:34.031952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.027140    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.041345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.040902    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.041345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.041020    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxmnk\" (UniqueName: \"kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.041345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.041150    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.041345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.041227    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.044454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.044432    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-76f654-serving-cert\""
Oct 30 01:04:34.163021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.159765    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.163021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.160040    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.163021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.160151    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxmnk\" (UniqueName: \"kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.163021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.160321    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.176909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.168742    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.176909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.172764    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.202903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.202295    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.206887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.203334    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxmnk\" (UniqueName: \"kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk\") pod \"maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.355676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.355593    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:34.833904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.833784    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" event={"ID":"421db701-5ef1-4a24-8fc6-342e88a351bf","Type":"ContainerStarted","Data":"e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"}
Oct 30 01:04:34.894654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.894577    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podStartSLOduration=5.89455723 podStartE2EDuration="5.89455723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:34.889146266 +0000 UTC m=+1382.918456498" watchObservedRunningTime="2026-10-30 01:04:34.89455723 +0000 UTC m=+1382.923867464"
Oct 30 01:04:34.911608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.911575    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:34.944145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:34.944120    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"]
Oct 30 01:04:34.944245 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:04:34.944157    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d9d96f_59af_4a77_b8d3_71b3309cf224.slice/crio-97c6ae2f88212ed399b1e77e8ddecaf2bbb7a07e2c35f3218f58a4be83b44965 WatchSource:0}: Error finding container 97c6ae2f88212ed399b1e77e8ddecaf2bbb7a07e2c35f3218f58a4be83b44965: Status 404 returned error can't find the container with id 97c6ae2f88212ed399b1e77e8ddecaf2bbb7a07e2c35f3218f58a4be83b44965
Oct 30 01:04:35.464626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:35.456111    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:35.722919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:35.720429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:35.830867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:35.823244    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:35.894930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:35.868898    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:35.896691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:35.895782    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:36.027982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.027897    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p" event={"ID":"74d9d96f-59af-4a77-b8d3-71b3309cf224","Type":"ContainerStarted","Data":"f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926"}
Oct 30 01:04:36.028158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.028020    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p" event={"ID":"74d9d96f-59af-4a77-b8d3-71b3309cf224","Type":"ContainerStarted","Data":"97c6ae2f88212ed399b1e77e8ddecaf2bbb7a07e2c35f3218f58a4be83b44965"}
Oct 30 01:04:36.030962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.028838    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:36.030962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.029269    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:36.036671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.036203    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:04:36.135244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.135187    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p" podStartSLOduration=2.6238584769999997 podStartE2EDuration="3.135168595s" totalImagesPullingTime="511.310118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:36.132724019 +0000 UTC m=+1384.162034269" watchObservedRunningTime="2026-10-30 01:04:36.135168595 +0000 UTC m=+1384.164478826"
Oct 30 01:04:36.213576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:36.213540    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:37.455060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:37.455014    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:37.577654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:37.577622    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:37.703596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:37.703520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:39.125434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:39.125398    4577 scope.go:122] "RemoveContainer" containerID="aef49c3cbb57b6e3c244410d1948e88af67cac482cd53a3ba63d86996b5fa3ad"
Oct 30 01:04:39.186449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:39.186350    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:39.203226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:39.202727    4577 scope.go:122] "RemoveContainer" containerID="7908f8a31d4028d92d55b5620db9eaefd083404cd06002d06408d7d12f999c76"
Oct 30 01:04:39.273991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:39.273965    4577 scope.go:122] "RemoveContainer" containerID="f6aaeda0bb85095b10f5e25ffd6d2ee3386d5810cf6c413a44144e92a2a57b47"
Oct 30 01:04:39.306083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:39.306000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:39.594930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:39.561660    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:41.300797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:41.300758    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:42.033433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.033398    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"]
Oct 30 01:04:42.046517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.046492    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78104af2-4cb7t"]
Oct 30 01:04:42.115359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.115323    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a49ae627-e9ff-4695-b47b-e043b89ff169" path="/var/lib/kubelet/pods/a49ae627-e9ff-4695-b47b-e043b89ff169/volumes"
Oct 30 01:04:42.136396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.136368    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:42.248961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.248927    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:42.651053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.650946    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"]
Oct 30 01:04:42.651525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.651224    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" containerID="cri-o://507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376" gracePeriod=30
Oct 30 01:04:42.661176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.661148    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"]
Oct 30 01:04:42.661372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.661351    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" containerID="cri-o://e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f" gracePeriod=30
Oct 30 01:04:42.677601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:42.677574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:43.094869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:43.088070    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:04:43.188827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:43.188792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:43.380510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:43.380424    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:04:43.380773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:43.380733    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:04:44.125813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:44.125777    4577 patch_prober.go:29] interesting pod/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:04:44.126231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:44.125830    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:04:44.899230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:44.899192    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:46.604265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:46.604232    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:47.981508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:47.981438    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:04:48.023057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023030    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023135    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023259    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023441    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023473    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023480    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.023529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023513    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023547    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023656    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.023831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.023729    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4\") pod \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\" (UID: \"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801\") "
Oct 30 01:04:48.024231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.024138    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.024627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.024601    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.024936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.024631    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.024936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.024814    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:48.024936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.024896    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.025127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.024949    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.026642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.026607    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.027312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.027285    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:48.027916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.027896    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "kube-api-access-frfl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:48.029527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.029499    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token" pod "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" (UID: "ec1cbd50-b3d0-4cd9-b519-4d1bdd543801"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:48.032467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.032446    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:04:48.056119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.056072    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:48.125729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.125687    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.125983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.125790    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.125983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.125928    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.125983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.125972    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.125996    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126032    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126071    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126096    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126117    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126205    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126301    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy\") pod \"4bad413b-e468-498a-8343-29be621eeeae\" (UID: \"4bad413b-e468-498a-8343-29be621eeeae\") "
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126385    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126641    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.126757    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127227    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127250    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127265    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bad413b-e468-498a-8343-29be621eeeae-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127279    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frfl4\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-kube-api-access-frfl4\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127295    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127308    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127321    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127334    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127347    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127360    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127374    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.127457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.127386    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.129068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.128966    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:48.129323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.129291    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:48.129415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.129341    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "kube-api-access-qtjk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:48.130463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.130429    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo" pod "4bad413b-e468-498a-8343-29be621eeeae" (UID: "4bad413b-e468-498a-8343-29be621eeeae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:48.170455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.170411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:48.179678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.179653    4577 generic.go:356] "Generic (PLEG): container finished" podID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerID="507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376" exitCode=0
Oct 30 01:04:48.179880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.179723    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"
Oct 30 01:04:48.179880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.179750    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" event={"ID":"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801","Type":"ContainerDied","Data":"507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376"}
Oct 30 01:04:48.179880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.179805    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp" event={"ID":"ec1cbd50-b3d0-4cd9-b519-4d1bdd543801","Type":"ContainerDied","Data":"744d98ab75fe7b0a574e6a8222ee023201a7813639378daa89ac0914b063ddd6"}
Oct 30 01:04:48.179880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.179832    4577 scope.go:122] "RemoveContainer" containerID="507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376"
Oct 30 01:04:48.181838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.181732    4577 generic.go:356] "Generic (PLEG): container finished" podID="4bad413b-e468-498a-8343-29be621eeeae" containerID="e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f" exitCode=0
Oct 30 01:04:48.181961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.181789    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"
Oct 30 01:04:48.182164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.181811    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" event={"ID":"4bad413b-e468-498a-8343-29be621eeeae","Type":"ContainerDied","Data":"e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f"}
Oct 30 01:04:48.182164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.182130    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4" event={"ID":"4bad413b-e468-498a-8343-29be621eeeae","Type":"ContainerDied","Data":"4baf1d78af1e89de951c906f55ac9cdfaaa0433ca73226bc22543c09466bdc1a"}
Oct 30 01:04:48.191806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.191785    4577 scope.go:122] "RemoveContainer" containerID="507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376"
Oct 30 01:04:48.192399 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:48.192377    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376\": container with ID starting with 507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376 not found: ID does not exist" containerID="507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376"
Oct 30 01:04:48.192510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.192408    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376"} err="failed to get container status \"507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376\": rpc error: code = NotFound desc = could not find container \"507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376\": container with ID starting with 507a4b4c9ddb0c47ec039af2b600f0d5e951c40aa7ca24a5831e60a0b40ed376 not found: ID does not exist"
Oct 30 01:04:48.192510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.192425    4577 scope.go:122] "RemoveContainer" containerID="e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f"
Oct 30 01:04:48.203298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.203279    4577 scope.go:122] "RemoveContainer" containerID="8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"
Oct 30 01:04:48.214070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.214054    4577 scope.go:122] "RemoveContainer" containerID="e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f"
Oct 30 01:04:48.214341 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:48.214318    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f\": container with ID starting with e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f not found: ID does not exist" containerID="e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f"
Oct 30 01:04:48.214422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.214359    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f"} err="failed to get container status \"e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f\": rpc error: code = NotFound desc = could not find container \"e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f\": container with ID starting with e50893f2af70d39be899861fab2120e3f324c803d25ea53a179c306fb308e92f not found: ID does not exist"
Oct 30 01:04:48.214422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.214385    4577 scope.go:122] "RemoveContainer" containerID="8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"
Oct 30 01:04:48.214649 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:48.214631    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d\": container with ID starting with 8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d not found: ID does not exist" containerID="8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"
Oct 30 01:04:48.214696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.214656    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d"} err="failed to get container status \"8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d\": rpc error: code = NotFound desc = could not find container \"8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d\": container with ID starting with 8518347f951d7fe80cbd5e75bc5cf4c95fc6da422d1fbd00631d2f718c1c974d not found: ID does not exist"
Oct 30 01:04:48.228792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.228769    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bad413b-e468-498a-8343-29be621eeeae-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.228911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.228795    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.228911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.228808    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bad413b-e468-498a-8343-29be621eeeae-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.228911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.228821    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtjk7\" (UniqueName: \"kubernetes.io/projected/4bad413b-e468-498a-8343-29be621eeeae-kube-api-access-qtjk7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:48.240257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.240237    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"]
Oct 30 01:04:48.244251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.244233    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-r9rp4"]
Oct 30 01:04:48.281859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.281823    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"]
Oct 30 01:04:48.286140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.286114    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9a8d85-openshift-default-84cb77f8df-gvrbp"]
Oct 30 01:04:48.517798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:48.517767    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:49.535524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:49.535490    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"]
Oct 30 01:04:49.535952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:49.535711    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy" containerID="cri-o://7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec" gracePeriod=30
Oct 30 01:04:50.152870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:50.151565    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bad413b-e468-498a-8343-29be621eeeae" path="/var/lib/kubelet/pods/4bad413b-e468-498a-8343-29be621eeeae/volumes"
Oct 30 01:04:50.180750 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:50.175538    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" path="/var/lib/kubelet/pods/ec1cbd50-b3d0-4cd9-b519-4d1bdd543801/volumes"
Oct 30 01:04:50.337874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:50.336948    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:51.897865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:51.896023    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:51.921943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:51.917932    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"]
Oct 30 01:04:51.921943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:51.918193    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" podUID="a9701959-bf30-41cd-b0a7-b156593ff26b" containerName="maas-api" containerID="cri-o://3042da501ca42027f33ddf5b430c77ad1adf4618193c1602d3d22c93c863a138" gracePeriod=30
Oct 30 01:04:52.499490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:52.499204    4577 generic.go:356] "Generic (PLEG): container finished" podID="a9701959-bf30-41cd-b0a7-b156593ff26b" containerID="3042da501ca42027f33ddf5b430c77ad1adf4618193c1602d3d22c93c863a138" exitCode=0
Oct 30 01:04:52.499490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:52.499363    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" event={"ID":"a9701959-bf30-41cd-b0a7-b156593ff26b","Type":"ContainerDied","Data":"3042da501ca42027f33ddf5b430c77ad1adf4618193c1602d3d22c93c863a138"}
Oct 30 01:04:52.651982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:52.651833    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:52.867316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:52.867284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:53.019885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.019765    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:53.311927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.292039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:53.338873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.333645    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:04:53.339267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.336121    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"]
Oct 30 01:04:53.339725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.339674    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy" containerID="cri-o://4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093" gracePeriod=30
Oct 30 01:04:53.402506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.402467    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca\") pod \"a9701959-bf30-41cd-b0a7-b156593ff26b\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") "
Oct 30 01:04:53.441872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.439300    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca" pod "a9701959-bf30-41cd-b0a7-b156593ff26b" (UID: "a9701959-bf30-41cd-b0a7-b156593ff26b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:53.441872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.439623    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\") pod \"a9701959-bf30-41cd-b0a7-b156593ff26b\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") "
Oct 30 01:04:53.441872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.440004    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs\") pod \"a9701959-bf30-41cd-b0a7-b156593ff26b\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") "
Oct 30 01:04:53.441872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.440231    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q\" (UniqueName: \"kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q\") pod \"a9701959-bf30-41cd-b0a7-b156593ff26b\" (UID: \"a9701959-bf30-41cd-b0a7-b156593ff26b\") "
Oct 30 01:04:53.441872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.441767    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9701959-bf30-41cd-b0a7-b156593ff26b-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:53.442701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.442663    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls" pod "a9701959-bf30-41cd-b0a7-b156593ff26b" (UID: "a9701959-bf30-41cd-b0a7-b156593ff26b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:53.443018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.442989    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q" pod "a9701959-bf30-41cd-b0a7-b156593ff26b" (UID: "a9701959-bf30-41cd-b0a7-b156593ff26b"). InnerVolumeSpecName "kube-api-access-brz2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:53.444486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.444461    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs" pod "a9701959-bf30-41cd-b0a7-b156593ff26b" (UID: "a9701959-bf30-41cd-b0a7-b156593ff26b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:53.482644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.482400    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"]
Oct 30 01:04:53.482833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.482684    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy" containerID="cri-o://8df9b9e02eb4ac3af9a0bdfc6c681b61b43879b5a5e032c75aa27884dfb67c13" gracePeriod=30
Oct 30 01:04:53.488210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.487790    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"]
Oct 30 01:04:53.488210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.488039    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy" containerID="cri-o://5bde485049c7d17a594d9403aa7622a5ee938a2417540eb0d40af1176482ca41" gracePeriod=30
Oct 30 01:04:53.526390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.526354    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:53.531915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.531880    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk" event={"ID":"a9701959-bf30-41cd-b0a7-b156593ff26b","Type":"ContainerDied","Data":"12ea449a923d686a290374e330495384b338017538d07b2af16b2de78c696bd6"}
Oct 30 01:04:53.532081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.531917    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"
Oct 30 01:04:53.532081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.531979    4577 scope.go:122] "RemoveContainer" containerID="3042da501ca42027f33ddf5b430c77ad1adf4618193c1602d3d22c93c863a138"
Oct 30 01:04:53.543777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.543750    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:53.543910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.543783    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9701959-bf30-41cd-b0a7-b156593ff26b-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:53.543910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.543798    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brz2q\" (UniqueName: \"kubernetes.io/projected/a9701959-bf30-41cd-b0a7-b156593ff26b-kube-api-access-brz2q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:53.655528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.655411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:53.688777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.688743    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"]
Oct 30 01:04:53.695823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:53.695782    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9b2c3f-6764d5dd-h4xwk"]
Oct 30 01:04:54.000940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.000909    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"]
Oct 30 01:04:54.001651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001631    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy"
Oct 30 01:04:54.001651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001648    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001660    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a9701959-bf30-41cd-b0a7-b156593ff26b" containerName="maas-api"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001666    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9701959-bf30-41cd-b0a7-b156593ff26b" containerName="maas-api"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001674    4577 cpu_manager.go:414] "containerMap: removing container" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001680    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001697    4577 cpu_manager.go:414] "containerMap: removing container" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001701    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001806    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec1cbd50-b3d0-4cd9-b519-4d1bdd543801" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001813    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9701959-bf30-41cd-b0a7-b156593ff26b" containerName="maas-api"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001819    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy"
Oct 30 01:04:54.001874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.001827    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bad413b-e468-498a-8343-29be621eeeae" containerName="istio-proxy"
Oct 30 01:04:54.005322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.005302    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.038938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.033524    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-11fa15-openshift-default-dockercfg-k9csr\""
Oct 30 01:04:54.039576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.039150    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"]
Oct 30 01:04:54.079897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.079432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:54.081010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.080969    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081017    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081150    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081196    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081282    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081400    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081455    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081498    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4gxc\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.081593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.081538    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.114126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.114099    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9701959-bf30-41cd-b0a7-b156593ff26b" path="/var/lib/kubelet/pods/a9701959-bf30-41cd-b0a7-b156593ff26b/volumes"
Oct 30 01:04:54.213359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.213310    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.222696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.222658    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.223067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.223043    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.223244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.223227    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.223577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.223542    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.236243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.236198    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.236551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.236530    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.236728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.236703    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4gxc\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.236886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.236869    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.237672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.237650    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.250360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.249232    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.250360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.224168    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.250360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.250300    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.250934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.224180    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.277732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.270874    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:54.301201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.286496    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.377881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.362787    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.455874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.452547    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.462261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.462190    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4gxc\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc\") pod \"e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") " pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.640431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.640400    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:54.895645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.895566    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"]
Oct 30 01:04:54.901261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.900936    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:54.921118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.921032    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-d4622a-serving-cert\""
Oct 30 01:04:54.927792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.927756    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"]
Oct 30 01:04:54.997161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:54.997124    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:55.006451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.006417    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwtmk\" (UniqueName: \"kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.007075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.007046    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.007209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.007153    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.007421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.007405    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.019065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.019041    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 01:04:55.090149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.089999    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"]
Oct 30 01:04:55.091264 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:04:55.091233    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adfe7e6_3179_4162_b94c_566f2c3ffc96.slice/crio-45f7c4e59fbdeb62ca3f39c19829b6635d2cafceb6526ca026d4ee9b03727bbc WatchSource:0}: Error finding container 45f7c4e59fbdeb62ca3f39c19829b6635d2cafceb6526ca026d4ee9b03727bbc: Status 404 returned error can't find the container with id 45f7c4e59fbdeb62ca3f39c19829b6635d2cafceb6526ca026d4ee9b03727bbc
Oct 30 01:04:55.093906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.093866    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:55.094119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.093957    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:55.094119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.094000    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:04:55.113161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113123    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113184    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113286    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113352    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113476    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113509    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113565    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113619    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.113734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.113690    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data\") pod \"753f43f0-4c9f-47a6-bf56-6a80eba29455\" (UID: \"753f43f0-4c9f-47a6-bf56-6a80eba29455\") "
Oct 30 01:04:55.114256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.114225    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:55.117058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.114610    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:55.117058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.114419    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:55.117058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.114646    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:55.117058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.114977    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:55.117058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.115148    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:55.118450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.118424    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:55.120863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.119076    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.120863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.119675    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwtmk\" (UniqueName: \"kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.120863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.120384    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.120863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.120448    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.120863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.120756    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.121311 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:55.120962    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-serving-cert: secret "maas-api-e2e-ipp-a-d4622a-serving-cert" not found
Oct 30 01:04:55.121311 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:55.121242    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls podName:ccd2c83f-436f-465f-95c9-bf74a557f12a nodeName:}" failed. No retries permitted until 2026-10-30 01:04:55.621202449 +0000 UTC m=+1403.650512675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls") pod "maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" (UID: "ccd2c83f-436f-465f-95c9-bf74a557f12a") : secret "maas-api-e2e-ipp-a-d4622a-serving-cert" not found
Oct 30 01:04:55.121866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.121830    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.121976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.121889    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.121976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.121905    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.121976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.121920    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.121976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.121970    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.122208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.121987    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/753f43f0-4c9f-47a6-bf56-6a80eba29455-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.123575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.123518    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.127545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.127512    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:55.128258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.128220    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:55.136951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.135050    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4" pod "753f43f0-4c9f-47a6-bf56-6a80eba29455" (UID: "753f43f0-4c9f-47a6-bf56-6a80eba29455"). InnerVolumeSpecName "kube-api-access-mnmx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:55.197922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.193158    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwtmk\" (UniqueName: \"kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.253898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.251095    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.253898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.251131    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/753f43f0-4c9f-47a6-bf56-6a80eba29455-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.253898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.251148    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnmx4\" (UniqueName: \"kubernetes.io/projected/753f43f0-4c9f-47a6-bf56-6a80eba29455-kube-api-access-mnmx4\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:55.286188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.284942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:55.672113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.672072    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.679607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.679576    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.720654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.720620    4577 generic.go:356] "Generic (PLEG): container finished" podID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerID="7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec" exitCode=0
Oct 30 01:04:55.720806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.720697    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"
Oct 30 01:04:55.720806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.720707    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" event={"ID":"753f43f0-4c9f-47a6-bf56-6a80eba29455","Type":"ContainerDied","Data":"7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec"}
Oct 30 01:04:55.720806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.720762    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522" event={"ID":"753f43f0-4c9f-47a6-bf56-6a80eba29455","Type":"ContainerDied","Data":"c03108d1e8447a6ef7137be6d78c647e8f75eb91888c5785d95f69a2d4642c5c"}
Oct 30 01:04:55.720806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.720792    4577 scope.go:122] "RemoveContainer" containerID="7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec"
Oct 30 01:04:55.723270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.723236    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" event={"ID":"6adfe7e6-3179-4162-b94c-566f2c3ffc96","Type":"ContainerStarted","Data":"3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"}
Oct 30 01:04:55.723366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.723291    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" event={"ID":"6adfe7e6-3179-4162-b94c-566f2c3ffc96","Type":"ContainerStarted","Data":"45f7c4e59fbdeb62ca3f39c19829b6635d2cafceb6526ca026d4ee9b03727bbc"}
Oct 30 01:04:55.734098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.733022    4577 scope.go:122] "RemoveContainer" containerID="7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec"
Oct 30 01:04:55.734098 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:55.733366    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec\": container with ID starting with 7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec not found: ID does not exist" containerID="7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec"
Oct 30 01:04:55.734098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.733399    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec"} err="failed to get container status \"7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec\": rpc error: code = NotFound desc = could not find container \"7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec\": container with ID starting with 7734dbcda281d2ed62d2be47432d59ed1349cc511fbe4bb2dbc1edc8430c44ec not found: ID does not exist"
Oct 30 01:04:55.785419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.785381    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"]
Oct 30 01:04:55.792381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.792349    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-4k522"]
Oct 30 01:04:55.826089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.825094    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:55.866602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:55.866520    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podStartSLOduration=2.866497884 podStartE2EDuration="2.866497884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:55.831371992 +0000 UTC m=+1403.860682223" watchObservedRunningTime="2026-10-30 01:04:55.866497884 +0000 UTC m=+1403.895808115"
Oct 30 01:04:56.165124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.165073    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" path="/var/lib/kubelet/pods/753f43f0-4c9f-47a6-bf56-6a80eba29455/volumes"
Oct 30 01:04:56.668984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.667915    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:56.679584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.679554    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:56.718774 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:04:56.718739    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd2c83f_436f_465f_95c9_bf74a557f12a.slice/crio-da996b60dbbfbbf3fee3e89332c888128ddf158ceaa90693705981e2b87a393a WatchSource:0}: Error finding container da996b60dbbfbbf3fee3e89332c888128ddf158ceaa90693705981e2b87a393a: Status 404 returned error can't find the container with id da996b60dbbfbbf3fee3e89332c888128ddf158ceaa90693705981e2b87a393a
Oct 30 01:04:56.719010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.718988    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"]
Oct 30 01:04:56.733450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.733423    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" event={"ID":"ccd2c83f-436f-465f-95c9-bf74a557f12a","Type":"ContainerStarted","Data":"da996b60dbbfbbf3fee3e89332c888128ddf158ceaa90693705981e2b87a393a"}
Oct 30 01:04:56.734974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.734953    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:56.735826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.735807    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:04:56.976642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:56.976555    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:57.064869 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:57.063951    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:04:57.758872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:57.758347    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" event={"ID":"ccd2c83f-436f-465f-95c9-bf74a557f12a","Type":"ContainerStarted","Data":"f47e28fc8f82a8103c9993fdfd7eabfbe3f0ae1343202f8c658332e4c9235bab"}
Oct 30 01:04:57.758872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:57.758423    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:04:57.807317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:57.807256    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" podStartSLOduration=3.236575908 podStartE2EDuration="3.807238746s" totalImagesPullingTime="570.662838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:57.803511727 +0000 UTC m=+1405.832821957" watchObservedRunningTime="2026-10-30 01:04:57.807238746 +0000 UTC m=+1405.836548979"
Oct 30 01:04:58.083952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.074740    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"]
Oct 30 01:04:58.098168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.098134    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-87287c79-stlv4"]
Oct 30 01:04:58.116241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.116154    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="355e2206-9f05-456e-8d61-7b079a6b809c" path="/var/lib/kubelet/pods/355e2206-9f05-456e-8d61-7b079a6b809c/volumes"
Oct 30 01:04:58.373445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.373362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:58.499882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.498175    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:58.696574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.696546    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.784127    4577 generic.go:356] "Generic (PLEG): container finished" podID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerID="8df9b9e02eb4ac3af9a0bdfc6c681b61b43879b5a5e032c75aa27884dfb67c13" exitCode=0
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.784268    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" event={"ID":"7f89417c-ccc2-40ca-a13f-6014e47ea558","Type":"ContainerDied","Data":"8df9b9e02eb4ac3af9a0bdfc6c681b61b43879b5a5e032c75aa27884dfb67c13"}
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.786570    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.787183    4577 generic.go:356] "Generic (PLEG): container finished" podID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerID="4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093" exitCode=0
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.787393    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.787602    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" event={"ID":"2deeef02-5405-4d0c-baa7-4e6b9f9502fa","Type":"ContainerDied","Data":"4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093"}
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.787666    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf" event={"ID":"2deeef02-5405-4d0c-baa7-4e6b9f9502fa","Type":"ContainerDied","Data":"243e3a70491347643859515b95273b9884f24fd432b850e0ebc20802b0376afd"}
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.787700    4577 scope.go:122] "RemoveContainer" containerID="4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093"
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.792076    4577 generic.go:356] "Generic (PLEG): container finished" podID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerID="5bde485049c7d17a594d9403aa7622a5ee938a2417540eb0d40af1176482ca41" exitCode=0
Oct 30 01:04:58.794901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.792252    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" event={"ID":"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7","Type":"ContainerDied","Data":"5bde485049c7d17a594d9403aa7622a5ee938a2417540eb0d40af1176482ca41"}
Oct 30 01:04:58.805041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805002    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805073    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805249    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805308    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805361    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805431    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805493    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805634    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.805715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.805701    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s\") pod \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\" (UID: \"2deeef02-5405-4d0c-baa7-4e6b9f9502fa\") "
Oct 30 01:04:58.806534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.806514    4577 scope.go:122] "RemoveContainer" containerID="4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093"
Oct 30 01:04:58.806752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.806725    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.807129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.807103    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.807298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.807278    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.807396 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:04:58.807373    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093\": container with ID starting with 4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093 not found: ID does not exist" containerID="4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093"
Oct 30 01:04:58.807460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.807410    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093"} err="failed to get container status \"4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093\": rpc error: code = NotFound desc = could not find container \"4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093\": container with ID starting with 4994a5fe737ff9601bdb9cfe406bbdd3c28516249366c67e4b2e2fd773e6f093 not found: ID does not exist"
Oct 30 01:04:58.807535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.807489    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:58.807766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.807746    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.809314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.809266    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "kube-api-access-cc49s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:58.810307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.810282    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.810411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.810330    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:58.810634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.810608    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token" pod "2deeef02-5405-4d0c-baa7-4e6b9f9502fa" (UID: "2deeef02-5405-4d0c-baa7-4e6b9f9502fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:58.908120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.907822    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.908120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.907906    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.908120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.907967    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.908120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908076    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.908120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908101    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.908575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908145    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.908637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908571    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908894    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908975    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.909025    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket\") pod \"7f89417c-ccc2-40ca-a13f-6014e47ea558\" (UID: \"7f89417c-ccc2-40ca-a13f-6014e47ea558\") "
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.910211    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.910393    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.908833    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.910887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.910818    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911293    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911314    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911328    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911341    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911354    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911353    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "kube-api-access-pf2rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911370    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911384    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911397    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911410    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc49s\" (UniqueName: \"kubernetes.io/projected/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-kube-api-access-cc49s\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911424    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911428    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911438    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911450    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911464    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f89417c-ccc2-40ca-a13f-6014e47ea558-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.911874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911477    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2deeef02-5405-4d0c-baa7-4e6b9f9502fa-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:58.912391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.911837    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:58.913833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.913771    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token" pod "7f89417c-ccc2-40ca-a13f-6014e47ea558" (UID: "7f89417c-ccc2-40ca-a13f-6014e47ea558"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:58.945763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:58.945733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:04:59.024875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.022153    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf2rt\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-kube-api-access-pf2rt\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.024875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.022187    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.024875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.022205    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.024875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.022220    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f89417c-ccc2-40ca-a13f-6014e47ea558-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.162018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.161890    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"]
Oct 30 01:04:59.166337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.166311    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-7b8tf"]
Oct 30 01:04:59.270624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.270599    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 01:04:59.327653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.327623    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.327904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.327715    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.327904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.327896    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.327936    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.327962    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328003    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328049    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328054    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:59.328323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328115    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328145    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data\") pod \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\" (UID: \"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7\") "
Oct 30 01:04:59.328432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328348    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:59.328605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328568    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:59.328943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328920    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.329032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328946    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.329032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.328959    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.329192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.329168    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:59.329354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.329329    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:59.332421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.332372    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:59.332421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.332372    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:59.332576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.332550    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "kube-api-access-zjww2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:59.332637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.332580    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo" pod "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" (UID: "ef5f4dc1-6714-48ac-897a-faa1cc6e29e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:59.430320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.430232    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjww2\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-kube-api-access-zjww2\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.430320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.430269    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.430320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.430282    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.430320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.430293    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.430320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.430304    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.430320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.430315    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:04:59.802225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.802197    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"
Oct 30 01:04:59.802660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.802196    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk" event={"ID":"7f89417c-ccc2-40ca-a13f-6014e47ea558","Type":"ContainerDied","Data":"6d008e22b5c39c54ed31bfe4e5935c70f2aa25aba23096bc8fb5475d4aa3b029"}
Oct 30 01:04:59.802660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.802358    4577 scope.go:122] "RemoveContainer" containerID="8df9b9e02eb4ac3af9a0bdfc6c681b61b43879b5a5e032c75aa27884dfb67c13"
Oct 30 01:04:59.806091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.806063    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"
Oct 30 01:04:59.812893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.809088    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz" event={"ID":"ef5f4dc1-6714-48ac-897a-faa1cc6e29e7","Type":"ContainerDied","Data":"2c96986c87b07117ea22923df46f31b26ec1fb36b8890b268b7858cf45a4b68e"}
Oct 30 01:04:59.815783 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.815759    4577 scope.go:122] "RemoveContainer" containerID="5bde485049c7d17a594d9403aa7622a5ee938a2417540eb0d40af1176482ca41"
Oct 30 01:04:59.861271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.861238    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"]
Oct 30 01:04:59.867994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.867957    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-xm2lk"]
Oct 30 01:04:59.911317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.911281    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"]
Oct 30 01:04:59.929702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:04:59.929671    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-k9hqz"]
Oct 30 01:05:00.162201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.162068    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:00.303870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.297666    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:00.303870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.299578    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" path="/var/lib/kubelet/pods/2deeef02-5405-4d0c-baa7-4e6b9f9502fa/volumes"
Oct 30 01:05:00.303870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.300446    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" path="/var/lib/kubelet/pods/7f89417c-ccc2-40ca-a13f-6014e47ea558/volumes"
Oct 30 01:05:00.306689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.306654    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" path="/var/lib/kubelet/pods/ef5f4dc1-6714-48ac-897a-faa1cc6e29e7/volumes"
Oct 30 01:05:00.310678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.310637    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"]
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340497    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340527    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340579    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340588    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340622    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340629    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340655    4577 cpu_manager.go:414] "containerMap: removing container" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.340662    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.341015    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2deeef02-5405-4d0c-baa7-4e6b9f9502fa" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.341052    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef5f4dc1-6714-48ac-897a-faa1cc6e29e7" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.341070    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="753f43f0-4c9f-47a6-bf56-6a80eba29455" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.341088    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f89417c-ccc2-40ca-a13f-6014e47ea558" containerName="istio-proxy"
Oct 30 01:05:00.357868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.355438    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"]
Oct 30 01:05:00.363720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.361730    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.381272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.377022    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"]
Oct 30 01:05:00.381272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.377072    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"]
Oct 30 01:05:00.381272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.377626    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.384275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.384029    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-11fa15\""
Oct 30 01:05:00.384275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.384235    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-11fa15-dockercfg-6x7pd\""
Oct 30 01:05:00.496934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.473883    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.516910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.505293    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt54l\" (UniqueName: \"kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l\") pod \"payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.580047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.580007    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rzh8\" (UniqueName: \"kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8\") pod \"payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.580334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.580307    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.580775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.557011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:00.683004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.682965    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt54l\" (UniqueName: \"kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l\") pod \"payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.683258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.683239    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rzh8\" (UniqueName: \"kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8\") pod \"payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.683416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.683400    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.683632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.683614    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.703306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.684787    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.719078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.706404    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.719392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.714459    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:00.841000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.840873    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt54l\" (UniqueName: \"kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l\") pod \"payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:00.853798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.843237    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rzh8\" (UniqueName: \"kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8\") pod \"payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:00.874834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.874735    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:00.989491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:00.989388    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:01.003127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.003086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:01.150920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.132274    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:01.524150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.524120    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"]
Oct 30 01:05:01.693321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.693281    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"]
Oct 30 01:05:01.694170 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:01.694133    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932a7988_cdb7_47df_bbd5_29f2d0929804.slice/crio-2710b291a142ac1fc4691cadd764b31716005063c561fdd78fc36af655b01f86 WatchSource:0}: Error finding container 2710b291a142ac1fc4691cadd764b31716005063c561fdd78fc36af655b01f86: Status 404 returned error can't find the container with id 2710b291a142ac1fc4691cadd764b31716005063c561fdd78fc36af655b01f86
Oct 30 01:05:01.889043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.888881    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7" event={"ID":"dad28836-2b30-4803-9bb7-fa02c25075fa","Type":"ContainerStarted","Data":"d4ad16ff1891b06cdfddc47b9215fd9d2a1b696e8fc18e17db5cfd6417a0a983"}
Oct 30 01:05:01.889043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.888979    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:01.889043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.889002    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7" event={"ID":"dad28836-2b30-4803-9bb7-fa02c25075fa","Type":"ContainerStarted","Data":"43c4c826bc16a9255a27aaceceb6d8a345c7eefdd803d0e0793b60bf0c61fafe"}
Oct 30 01:05:01.895485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.895450    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk" event={"ID":"932a7988-cdb7-47df-bbd5-29f2d0929804","Type":"ContainerStarted","Data":"daf87e4392ff0ba72dd5f13024a2d044a7b9fde34fc53131fe080b3a84e9097f"}
Oct 30 01:05:01.895635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.895513    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk" event={"ID":"932a7988-cdb7-47df-bbd5-29f2d0929804","Type":"ContainerStarted","Data":"2710b291a142ac1fc4691cadd764b31716005063c561fdd78fc36af655b01f86"}
Oct 30 01:05:01.895635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.895544    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:01.937004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.936943    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7" podStartSLOduration=1.9369251410000001 podStartE2EDuration="1.936925141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:01.931610952 +0000 UTC m=+1409.960921182" watchObservedRunningTime="2026-10-30 01:05:01.936925141 +0000 UTC m=+1409.966235374"
Oct 30 01:05:01.979016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:01.978965    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk" podStartSLOduration=1.978951812 podStartE2EDuration="1.978951812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:01.976536605 +0000 UTC m=+1410.005846836" watchObservedRunningTime="2026-10-30 01:05:01.978951812 +0000 UTC m=+1410.008262042"
Oct 30 01:05:02.815626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:02.815595    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:03.720621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.720592    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"]
Oct 30 01:05:03.724127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.724106    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.751942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.751634    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"]
Oct 30 01:05:03.831185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831089    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831262    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831316    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831442    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831489    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831559    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831604    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831630    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.831830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.831803    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjpp\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.925273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.925181    4577 generic.go:356] "Generic (PLEG): container finished" podID="765e057d-2b90-442f-bec5-78841a2d9619" containerID="421e190c7c7910cb2d8389f448893e5260779a1c21b9826fcea0f3d883a93b4d" exitCode=0
Oct 30 01:05:03.925273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.925223    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" event={"ID":"765e057d-2b90-442f-bec5-78841a2d9619","Type":"ContainerDied","Data":"421e190c7c7910cb2d8389f448893e5260779a1c21b9826fcea0f3d883a93b4d"}
Oct 30 01:05:03.925747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.925728    4577 scope.go:122] "RemoveContainer" containerID="421e190c7c7910cb2d8389f448893e5260779a1c21b9826fcea0f3d883a93b4d"
Oct 30 01:05:03.927499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.927466    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:03.927608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.927551    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:03.927608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.927586    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:03.932777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.932755    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.932905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.932803    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.932965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.932923    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.932972    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933034    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933066    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933129    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjpp\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933212    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933513    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933616    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933684    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.933801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.933723    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.934264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.934242    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.935931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.935911    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.936652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.936631    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.957858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.957809    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:03.957986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:03.957886    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjpp\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp\") pod \"e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") " pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:04.067102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.067056    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:04.467625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.467533    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"]
Oct 30 01:05:04.472450 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:04.472420    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75c5f9f_7df6_4350_907d_f4b412ed4318.slice/crio-d30d4a18299596d50179a51b9c81b55c8d1b306c6ff6215f6fae955a62528656 WatchSource:0}: Error finding container d30d4a18299596d50179a51b9c81b55c8d1b306c6ff6215f6fae955a62528656: Status 404 returned error can't find the container with id d30d4a18299596d50179a51b9c81b55c8d1b306c6ff6215f6fae955a62528656
Oct 30 01:05:04.475683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.475635    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:04.475795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.475731    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:04.475795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.475773    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:04.584180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.584147    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"]
Oct 30 01:05:04.584443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.584419    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy" containerID="cri-o://376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7" gracePeriod=30
Oct 30 01:05:04.801670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.801537    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:04.855758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.843518    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:05:04.912545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.912357    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:04.937578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.937547    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" event={"ID":"765e057d-2b90-442f-bec5-78841a2d9619","Type":"ContainerStarted","Data":"9df1a3a8bd3c8d07f81316f28b07045e4c7d6187e67633706a1eb7d1236392c9"}
Oct 30 01:05:04.940718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.940681    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" event={"ID":"a75c5f9f-7df6-4350-907d-f4b412ed4318","Type":"ContainerStarted","Data":"e0dbbee626eae7d03c9df408ab074aef3a73885dc77bd2debc267af5647c7d91"}
Oct 30 01:05:04.940915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:04.940747    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" event={"ID":"a75c5f9f-7df6-4350-907d-f4b412ed4318","Type":"ContainerStarted","Data":"d30d4a18299596d50179a51b9c81b55c8d1b306c6ff6215f6fae955a62528656"}
Oct 30 01:05:05.086412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:05.086314    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:05.087565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:05.087530    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:05.087678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:05.087590    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:05.129291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:05.129251    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:05:05.130297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:05.130276    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:05.130413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:05.130320    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:06.071002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:06.070963    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:06.071633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:06.071594    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:06.130414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:06.130379    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:06.130587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:06.130434    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:06.329477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:06.329376    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:05:06.891896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:06.880663    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:07.107874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:07.093402    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:07.107874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:07.093464    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:07.158191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:07.152807    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:07.158191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:07.152885    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:07.534905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:07.525721    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:08.079916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:08.070988    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:08.079916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:08.071049    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:08.144652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:08.140726    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:08.144652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:08.140796    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:08.792871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:08.791863    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:08.920870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:08.913442    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:09.085928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:09.085310    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:09.085928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:09.085388    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:09.130134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:09.130098    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:09.130316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:09.130158    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:10.079874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.079383    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:10.079874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.079456    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:10.137963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.137258    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:10.137963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.137341    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:10.317754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.317725    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 01:05:10.323758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.323729    4577 generic.go:356] "Generic (PLEG): container finished" podID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerID="376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7" exitCode=0
Oct 30 01:05:10.324234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.324207    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" event={"ID":"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2","Type":"ContainerDied","Data":"376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7"}
Oct 30 01:05:10.324427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.324407    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn" event={"ID":"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2","Type":"ContainerDied","Data":"3b0fc8b43d9f6bc50bb8019b774f167b967f60d270ec976a4399bc39f51f464d"}
Oct 30 01:05:10.324572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.324557    4577 scope.go:122] "RemoveContainer" containerID="376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7"
Oct 30 01:05:10.324837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.324820    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"
Oct 30 01:05:10.409414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.409383    4577 scope.go:122] "RemoveContainer" containerID="376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7"
Oct 30 01:05:10.411061 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:10.411036    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7\": container with ID starting with 376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7 not found: ID does not exist" containerID="376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7"
Oct 30 01:05:10.411331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.411291    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7"} err="failed to get container status \"376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7\": rpc error: code = NotFound desc = could not find container \"376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7\": container with ID starting with 376cc651ebc8fbfc716bac897aa92c07df90fdd7ad8fdba79efd9bffc78a47c7 not found: ID does not exist"
Oct 30 01:05:10.440159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.440119    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.440394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.440228    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.440394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.440272    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.440932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.440890    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.441054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.440993    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.441118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.441093    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.441175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.441142    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.442999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.442960    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.443242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.443226    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token\") pod \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\" (UID: \"7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2\") "
Oct 30 01:05:10.444625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.444599    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "kube-api-access-f4vlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:10.460397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.460197    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:10.471539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.462990    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:10.471539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.467950    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:10.474891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.472046    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:10.483914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.475164    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:10.527154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.527111    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:10.528307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.528241    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podStartSLOduration=7.528220174 podStartE2EDuration="7.528220174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:05.086283915 +0000 UTC m=+1413.115594173" watchObservedRunningTime="2026-10-30 01:05:10.528220174 +0000 UTC m=+1418.557530407"
Oct 30 01:05:10.530334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.530300    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:10.532183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.532158    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy" pod "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" (UID: "7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:10.630876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.546012    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631147    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631285    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631376    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631465    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631550    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631634    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631723    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.631823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.631812    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4vlp\" (UniqueName: \"kubernetes.io/projected/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2-kube-api-access-f4vlp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:10.723809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.723716    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"]
Oct 30 01:05:10.729420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.729393    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-w54wn"]
Oct 30 01:05:10.744736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:10.744710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:11.077870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:11.077269    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:05:11.077870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:11.077347    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:05:11.129353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:11.129317    4577 patch_prober.go:29] interesting pod/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 01:05:11.129755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:11.129368    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 01:05:11.990897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:11.990234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:12.084871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.074423    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:12.144909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.140463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:12.172748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.172712    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" path="/var/lib/kubelet/pods/7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2/volumes"
Oct 30 01:05:12.173563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.173542    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:12.173892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.173827    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:05:12.174151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.174131    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:05:12.177168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.177146    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:05:12.627869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.624251    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:12.979927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.971293    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:12.996875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:12.980140    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:13.908242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:13.908210    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"]
Oct 30 01:05:13.908760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:13.908484    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy" containerID="cri-o://20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7" gracePeriod=30
Oct 30 01:05:13.924074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:13.924042    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"]
Oct 30 01:05:13.924501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:13.924470    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy" containerID="cri-o://e0bfdcb11e960cd7169f39efe33aee0957e3cc8daf7da5a18c0db5333a5c4f5c" gracePeriod=30
Oct 30 01:05:14.464943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:14.449579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:15.576475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:15.576434    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:15.706388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:15.706352    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:15.901976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:15.900588    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"]
Oct 30 01:05:15.901976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:15.900908    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy" containerID="cri-o://e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb" gracePeriod=30
Oct 30 01:05:15.928550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:15.928518    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:16.379088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.311427    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:16.561908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.561247    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:16.632879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.632771    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"]
Oct 30 01:05:16.634425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.634388    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy"
Oct 30 01:05:16.634578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.634565    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy"
Oct 30 01:05:16.634917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.634904    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7aaea6a7-d2fa-4d40-9a9d-cb46a3044cb2" containerName="istio-proxy"
Oct 30 01:05:16.639229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.639209    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"]
Oct 30 01:05:16.639391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.639301    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.681976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.679250    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-11fa15-serving-cert\""
Oct 30 01:05:16.692822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.692787    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:16.709971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.709945    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.710153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.709995    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.710153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.710139    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.710258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.710245    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswvw\" (UniqueName: \"kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.811320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.811291    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.811497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.811339    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.811497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.811419    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.811607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.811502    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswvw\" (UniqueName: \"kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.812295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.812217    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.814039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.814017    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.814189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.814166    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.816432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.816405    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:16.836657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.836627    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswvw\" (UniqueName: \"kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw\") pod \"maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:16.988428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:16.988383    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:17.324049 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:17.324007    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4811ec2_ed4f_4afa_9e94_a6e33b9624b8.slice/crio-a2f0d4421472e65d0b04210196ebfa555335b2d997c0fb85f57285bd20e90dad WatchSource:0}: Error finding container a2f0d4421472e65d0b04210196ebfa555335b2d997c0fb85f57285bd20e90dad: Status 404 returned error can't find the container with id a2f0d4421472e65d0b04210196ebfa555335b2d997c0fb85f57285bd20e90dad
Oct 30 01:05:17.324329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:17.324303    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"]
Oct 30 01:05:17.325832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:17.325812    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:05:17.659641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:17.659434    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr" event={"ID":"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8","Type":"ContainerStarted","Data":"a2f0d4421472e65d0b04210196ebfa555335b2d997c0fb85f57285bd20e90dad"}
Oct 30 01:05:18.180644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.180591    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:18.304736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.304677    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:18.692790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.692752    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr" event={"ID":"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8","Type":"ContainerStarted","Data":"20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2"}
Oct 30 01:05:18.694558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.694535    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:18.744639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.744576    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr" podStartSLOduration=2.390400322 podStartE2EDuration="2.744558242s" totalImagesPullingTime="354.15792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:18.74407797 +0000 UTC m=+1426.773388202" watchObservedRunningTime="2026-10-30 01:05:18.744558242 +0000 UTC m=+1426.773868474"
Oct 30 01:05:18.805550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.805518    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:18.927392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:18.927350    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:19.597887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.597391    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:19.644050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.644023    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 01:05:19.703663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.703634    4577 generic.go:356] "Generic (PLEG): container finished" podID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerID="e0bfdcb11e960cd7169f39efe33aee0957e3cc8daf7da5a18c0db5333a5c4f5c" exitCode=0
Oct 30 01:05:19.704018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.703704    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" event={"ID":"cc6b4213-198c-4ba3-b56a-09fe015ae4d8","Type":"ContainerDied","Data":"e0bfdcb11e960cd7169f39efe33aee0957e3cc8daf7da5a18c0db5333a5c4f5c"}
Oct 30 01:05:19.705358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.705334    4577 generic.go:356] "Generic (PLEG): container finished" podID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerID="20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7" exitCode=0
Oct 30 01:05:19.705466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.705400    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"
Oct 30 01:05:19.705466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.705415    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" event={"ID":"cc09a1df-6c62-4f2a-937f-c68a4c95852c","Type":"ContainerDied","Data":"20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7"}
Oct 30 01:05:19.705466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.705458    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l" event={"ID":"cc09a1df-6c62-4f2a-937f-c68a4c95852c","Type":"ContainerDied","Data":"3dd93c8d203cd3e2cb13b9a0cc1a15c9258d48e972b540a2d3706c9ac3538ba6"}
Oct 30 01:05:19.705570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.705482    4577 scope.go:122] "RemoveContainer" containerID="20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7"
Oct 30 01:05:19.712528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.712507    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:19.786219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.772344    4577 scope.go:122] "RemoveContainer" containerID="20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7"
Oct 30 01:05:19.786219 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:19.773726    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7\": container with ID starting with 20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7 not found: ID does not exist" containerID="20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7"
Oct 30 01:05:19.786219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.773826    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7"} err="failed to get container status \"20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7\": rpc error: code = NotFound desc = could not find container \"20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7\": container with ID starting with 20b5d2e1e1e934551121f1b87d626bb6875b622e426feb6fbec5e67b9ac4c4e7 not found: ID does not exist"
Oct 30 01:05:19.790512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790480    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 01:05:19.790660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790496    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.790660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790616    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.790795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790662    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.790795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790717    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.790932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790872    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.790932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.790909    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.791046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.791036    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.791282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.791256    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.791401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.791305    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket\") pod \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\" (UID: \"cc09a1df-6c62-4f2a-937f-c68a4c95852c\") "
Oct 30 01:05:19.791638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.791612    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.791750 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.791714    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.791958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.791937    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.792348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.792326    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.792414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.792355    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.792414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.792371    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.794013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.793985    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.794350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.794328    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:19.795318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.795294    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:19.795417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.795369    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "kube-api-access-pvfrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:19.796308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.796282    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:19.797008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.796989    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy" pod "cc09a1df-6c62-4f2a-937f-c68a4c95852c" (UID: "cc09a1df-6c62-4f2a-937f-c68a4c95852c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.904909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.904876    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.904930    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.904963    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.904988    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.905127    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.905160    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.905208    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.905271    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.905422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.905410    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo\") pod \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\" (UID: \"cc6b4213-198c-4ba3-b56a-09fe015ae4d8\") "
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.905993    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.906009    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvfrr\" (UniqueName: \"kubernetes.io/projected/cc09a1df-6c62-4f2a-937f-c68a4c95852c-kube-api-access-pvfrr\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.906019    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.906028    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.906036    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.906043    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc09a1df-6c62-4f2a-937f-c68a4c95852c-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.906574    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.907268    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.907455    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.907510    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.907558    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.909588    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:19.910010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.909968    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:19.912138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.912112    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:19.912572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:19.912549    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff" pod "cc6b4213-198c-4ba3-b56a-09fe015ae4d8" (UID: "cc6b4213-198c-4ba3-b56a-09fe015ae4d8"). InnerVolumeSpecName "kube-api-access-4vvff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007675    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007713    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007729    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007743    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007757    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007769    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007782    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vvff\" (UniqueName: \"kubernetes.io/projected/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-kube-api-access-4vvff\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007799    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.007879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.007811    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc6b4213-198c-4ba3-b56a-09fe015ae4d8-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:20.038170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.038091    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:20.097869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.092914    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"]
Oct 30 01:05:20.195872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.183593    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9fh6l"]
Oct 30 01:05:20.731032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.730998    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"
Oct 30 01:05:20.731032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.730997    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96" event={"ID":"cc6b4213-198c-4ba3-b56a-09fe015ae4d8","Type":"ContainerDied","Data":"85ea8d4f04d434ef1ba399301d499ffdfa9a7439fe33cb9421fb048ee20a140c"}
Oct 30 01:05:20.731631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.731122    4577 scope.go:122] "RemoveContainer" containerID="e0bfdcb11e960cd7169f39efe33aee0957e3cc8daf7da5a18c0db5333a5c4f5c"
Oct 30 01:05:20.805693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.805663    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"]
Oct 30 01:05:20.813396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.813363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:20.817500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:20.817478    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-cts96"]
Oct 30 01:05:21.199770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.199747    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266010    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266096    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266133    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266183    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266220    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266325    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266384    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266437    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.266868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.266488    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert\") pod \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\" (UID: \"fe8e2ee4-1059-413c-90d7-feaf02e70ed3\") "
Oct 30 01:05:21.267872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.267560    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:21.267872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.267742    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:21.268067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.267955    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:21.268395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.268371    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:21.268685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.268663    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:21.272375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.272308    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:21.272482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.272451    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "kube-api-access-qlglj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:21.274471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.273781    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:21.274471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.273962    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token" pod "fe8e2ee4-1059-413c-90d7-feaf02e70ed3" (UID: "fe8e2ee4-1059-413c-90d7-feaf02e70ed3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:21.368165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368123    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368156    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368168    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368180    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368192    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlglj\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-kube-api-access-qlglj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368207    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368218    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368229    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.368415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.368241    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e2ee4-1059-413c-90d7-feaf02e70ed3-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:21.646378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.646287    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"]
Oct 30 01:05:21.646570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.646515    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" containerID="cri-o://c444776022655decff217e1db0c5478fd0a65caabbdb79d1b1f77d9b41271494" gracePeriod=30
Oct 30 01:05:21.653725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.653663    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"]
Oct 30 01:05:21.658896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.654049    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" containerID="cri-o://1278349d97c1ec3f3c5dacc8df094efbfc2afee80b932dfcf5a0954299a3f6f9" gracePeriod=30
Oct 30 01:05:21.661990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.661966    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"]
Oct 30 01:05:21.662213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.662188    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy" containerID="cri-o://dfdefb346870153b888521158e493fec84db4da5226642ab7427fc0bf749b3ed" gracePeriod=30
Oct 30 01:05:21.745385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.745353    4577 generic.go:356] "Generic (PLEG): container finished" podID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerID="e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb" exitCode=0
Oct 30 01:05:21.745890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.745422    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"
Oct 30 01:05:21.745890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.745508    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" event={"ID":"fe8e2ee4-1059-413c-90d7-feaf02e70ed3","Type":"ContainerDied","Data":"e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb"}
Oct 30 01:05:21.745890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.745547    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph" event={"ID":"fe8e2ee4-1059-413c-90d7-feaf02e70ed3","Type":"ContainerDied","Data":"f8d4fd10cc3244db42ba8bfc804c62a691dbb64f611495c6b031fc00e9b94765"}
Oct 30 01:05:21.745890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.745572    4577 scope.go:122] "RemoveContainer" containerID="e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb"
Oct 30 01:05:21.770605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.770573    4577 scope.go:122] "RemoveContainer" containerID="65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"
Oct 30 01:05:21.785069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.785042    4577 scope.go:122] "RemoveContainer" containerID="e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb"
Oct 30 01:05:21.785564 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:21.785533    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb\": container with ID starting with e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb not found: ID does not exist" containerID="e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb"
Oct 30 01:05:21.785688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.785575    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb"} err="failed to get container status \"e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb\": rpc error: code = NotFound desc = could not find container \"e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb\": container with ID starting with e4be2add2aebf511e41810d92909672321033d52772a7b3129ed16880bab0cfb not found: ID does not exist"
Oct 30 01:05:21.785688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.785599    4577 scope.go:122] "RemoveContainer" containerID="65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"
Oct 30 01:05:21.785992 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:21.785966    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1\": container with ID starting with 65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1 not found: ID does not exist" containerID="65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"
Oct 30 01:05:21.786107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.786002    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1"} err="failed to get container status \"65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1\": rpc error: code = NotFound desc = could not find container \"65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1\": container with ID starting with 65e9437ad7009a3b5dfb9e8d8b25855e224a894eb3be17b87bd3ad46c17394c1 not found: ID does not exist"
Oct 30 01:05:21.824447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.824412    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:21.846100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.846065    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"]
Oct 30 01:05:21.852797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:21.852766    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8363a40b-openshift-default-677fff8ff7-58hph"]
Oct 30 01:05:22.071079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.071014    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:05:22.071304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.071114    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:05:22.116484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.116450    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" path="/var/lib/kubelet/pods/cc09a1df-6c62-4f2a-937f-c68a4c95852c/volumes"
Oct 30 01:05:22.117098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.117076    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" path="/var/lib/kubelet/pods/cc6b4213-198c-4ba3-b56a-09fe015ae4d8/volumes"
Oct 30 01:05:22.117624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.117605    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" path="/var/lib/kubelet/pods/fe8e2ee4-1059-413c-90d7-feaf02e70ed3/volumes"
Oct 30 01:05:22.787523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.787484    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:05:22.788030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:22.787551    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:05:23.511205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:23.511164    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:24.809965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:24.809929    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:05:24.810359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:24.809998    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:05:25.242656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.242625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:25.737397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.737371    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:05:25.797047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.797013    4577 generic.go:356] "Generic (PLEG): container finished" podID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerID="e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2" exitCode=0
Oct 30 01:05:25.797241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.797088    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" event={"ID":"421db701-5ef1-4a24-8fc6-342e88a351bf","Type":"ContainerDied","Data":"e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"}
Oct 30 01:05:25.797671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.797643    4577 scope.go:122] "RemoveContainer" containerID="e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"
Oct 30 01:05:25.800035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.800002    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:25.800142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.800091    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:25.800204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:25.800140    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:26.809758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.809724    4577 generic.go:356] "Generic (PLEG): container finished" podID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerID="c444776022655decff217e1db0c5478fd0a65caabbdb79d1b1f77d9b41271494" exitCode=0
Oct 30 01:05:26.810096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.809798    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" event={"ID":"489295d0-136f-47e6-9faf-4ebc614d6eb7","Type":"ContainerDied","Data":"c444776022655decff217e1db0c5478fd0a65caabbdb79d1b1f77d9b41271494"}
Oct 30 01:05:26.811539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.811517    4577 generic.go:356] "Generic (PLEG): container finished" podID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerID="dfdefb346870153b888521158e493fec84db4da5226642ab7427fc0bf749b3ed" exitCode=0
Oct 30 01:05:26.811708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.811654    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" event={"ID":"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c","Type":"ContainerDied","Data":"dfdefb346870153b888521158e493fec84db4da5226642ab7427fc0bf749b3ed"}
Oct 30 01:05:26.813230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.813153    4577 generic.go:356] "Generic (PLEG): container finished" podID="ec1fb88b-9674-4709-aca3-445a57945d83" containerID="1278349d97c1ec3f3c5dacc8df094efbfc2afee80b932dfcf5a0954299a3f6f9" exitCode=0
Oct 30 01:05:26.813352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.813225    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" event={"ID":"ec1fb88b-9674-4709-aca3-445a57945d83","Type":"ContainerDied","Data":"1278349d97c1ec3f3c5dacc8df094efbfc2afee80b932dfcf5a0954299a3f6f9"}
Oct 30 01:05:26.815574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.815282    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" event={"ID":"421db701-5ef1-4a24-8fc6-342e88a351bf","Type":"ContainerStarted","Data":"3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec"}
Oct 30 01:05:26.816383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.816291    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:05:26.816573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.816552    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 01:05:26.816617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:26.816598    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 01:05:27.021033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.020303    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 01:05:27.025061 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:27.025029    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:05:27.076871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.070000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.195527    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.195613    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.195676    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.195716    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.195833    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.195979    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.196031    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.196074    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.196884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.196128    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy\") pod \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\" (UID: \"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c\") "
Oct 30 01:05:27.224592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.224379    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.225206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.224901    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:27.260210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.230549    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.289072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.231205    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.289209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.231642    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.289311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.237650    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:27.289403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.238677    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:27.289484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.260493    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299213    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299246    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299260    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299273    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299286    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299298    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299311    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.315916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.299327    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.321761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.321718    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft" pod "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" (UID: "5bfd9074-ca1b-48e3-8aa1-02f5008fe44c"). InnerVolumeSpecName "kube-api-access-mbtft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:27.401233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.401191    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbtft\" (UniqueName: \"kubernetes.io/projected/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c-kube-api-access-mbtft\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.514181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.514096    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 01:05:27.593679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.593653    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 01:05:27.603971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.603941    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.604222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.604201    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.604608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.604587    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.604785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.604767    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.605016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.604990    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:27.605128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.605098    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.605263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.605245    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.605403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.605381    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.605523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.605506    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.605689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.605673    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.606231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.606213    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.606687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.606574    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.606860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.606826    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data\") pod \"ec1fb88b-9674-4709-aca3-445a57945d83\" (UID: \"ec1fb88b-9674-4709-aca3-445a57945d83\") "
Oct 30 01:05:27.607321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.607195    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.607442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.607414    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "kube-api-access-xt27w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:27.607542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.607490    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.608159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.608125    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:27.608508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.608474    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fb88b-9674-4709-aca3-445a57945d83-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.608647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.608633    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.608770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.608755    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.608904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.608890    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.609036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.609021    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.609166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.609152    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.609282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.609266    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt27w\" (UniqueName: \"kubernetes.io/projected/ec1fb88b-9674-4709-aca3-445a57945d83-kube-api-access-xt27w\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.609411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.609398    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fb88b-9674-4709-aca3-445a57945d83-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.611349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.611314    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo" pod "ec1fb88b-9674-4709-aca3-445a57945d83" (UID: "ec1fb88b-9674-4709-aca3-445a57945d83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:27.710689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.710664    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.710965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.710941    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711147    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711217    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711233    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.711349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711331    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711360    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.711418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711396    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711514    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.711570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711530    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711736    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:27.711832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711779    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.711832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.711817    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token\") pod \"489295d0-136f-47e6-9faf-4ebc614d6eb7\" (UID: \"489295d0-136f-47e6-9faf-4ebc614d6eb7\") "
Oct 30 01:05:27.712129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712024    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.712436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712416    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.712511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712438    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.712511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712447    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.712511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712456    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.712511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712464    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/489295d0-136f-47e6-9faf-4ebc614d6eb7-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.712511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.712477    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fb88b-9674-4709-aca3-445a57945d83-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.713656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.713633    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "kube-api-access-l8dtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:27.713966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.713948    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:27.714536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.714519    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:27.714588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.714566    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token" pod "489295d0-136f-47e6-9faf-4ebc614d6eb7" (UID: "489295d0-136f-47e6-9faf-4ebc614d6eb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:27.814211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.814118    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.814211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.814147    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.814211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.814157    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8dtb\" (UniqueName: \"kubernetes.io/projected/489295d0-136f-47e6-9faf-4ebc614d6eb7-kube-api-access-l8dtb\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.814211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.814166    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/489295d0-136f-47e6-9faf-4ebc614d6eb7-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:27.817148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.817119    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 01:05:27.817281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.817180    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 01:05:27.826655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.826631    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"
Oct 30 01:05:27.826801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.826643    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2" event={"ID":"489295d0-136f-47e6-9faf-4ebc614d6eb7","Type":"ContainerDied","Data":"83133f4b1d80d47638cf33d3b5caf19a0c9b4499d168a2af7e8aa017a359d722"}
Oct 30 01:05:27.826801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.826719    4577 scope.go:122] "RemoveContainer" containerID="c444776022655decff217e1db0c5478fd0a65caabbdb79d1b1f77d9b41271494"
Oct 30 01:05:27.828556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.828533    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f" event={"ID":"5bfd9074-ca1b-48e3-8aa1-02f5008fe44c","Type":"ContainerDied","Data":"4ca9d5632dd1d6a26a31ae9b3f962a755ecf20a7f9a5f4052a09d21f7b4f0e56"}
Oct 30 01:05:27.828556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.828549    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"
Oct 30 01:05:27.830476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.830455    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"
Oct 30 01:05:27.830601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.830496    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m" event={"ID":"ec1fb88b-9674-4709-aca3-445a57945d83","Type":"ContainerDied","Data":"524ed6d12c065c9b15edeb0289a5df9ec706b02e5019b23c641af78c1f34d32f"}
Oct 30 01:05:27.840769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.840750    4577 scope.go:122] "RemoveContainer" containerID="dfdefb346870153b888521158e493fec84db4da5226642ab7427fc0bf749b3ed"
Oct 30 01:05:27.884139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.884120    4577 scope.go:122] "RemoveContainer" containerID="1278349d97c1ec3f3c5dacc8df094efbfc2afee80b932dfcf5a0954299a3f6f9"
Oct 30 01:05:27.909009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.908975    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"]
Oct 30 01:05:27.913027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.912989    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-z2tx2"]
Oct 30 01:05:27.976729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.976692    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"]
Oct 30 01:05:27.980686 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:27.980655    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-5fz5f"]
Oct 30 01:05:28.012014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.011985    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"]
Oct 30 01:05:28.015458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.015437    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-8b82m"]
Oct 30 01:05:28.113100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.113024    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" path="/var/lib/kubelet/pods/489295d0-136f-47e6-9faf-4ebc614d6eb7/volumes"
Oct 30 01:05:28.113595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.113580    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" path="/var/lib/kubelet/pods/5bfd9074-ca1b-48e3-8aa1-02f5008fe44c/volumes"
Oct 30 01:05:28.114122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.114106    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" path="/var/lib/kubelet/pods/ec1fb88b-9674-4709-aca3-445a57945d83/volumes"
Oct 30 01:05:28.817461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.817421    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 01:05:28.817928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.817487    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 01:05:28.875868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:28.872750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:29.817086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:29.817049    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 01:05:29.817252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:29.817113    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 01:05:30.096835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.096761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:30.256792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.256753    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:30.702938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.702904    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"]
Oct 30 01:05:30.708154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.704219    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy"
Oct 30 01:05:30.712224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.712193    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy"
Oct 30 01:05:30.712454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.712440    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy"
Oct 30 01:05:30.722932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.722900    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy"
Oct 30 01:05:30.723164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723148    4577 cpu_manager.go:414] "containerMap: removing container" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy"
Oct 30 01:05:30.723275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723262    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy"
Oct 30 01:05:30.723386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723375    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy"
Oct 30 01:05:30.723468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723457    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy"
Oct 30 01:05:30.723557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723547    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy"
Oct 30 01:05:30.723642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723631    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy"
Oct 30 01:05:30.735193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.723716    4577 cpu_manager.go:414] "containerMap: removing container" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy"
Oct 30 01:05:30.735400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.735377    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy"
Oct 30 01:05:30.735547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.735535    4577 cpu_manager.go:414] "containerMap: removing container" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy"
Oct 30 01:05:30.735631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.735620    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy"
Oct 30 01:05:30.739455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.739427    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy"
Oct 30 01:05:30.739649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.739635    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bfd9074-ca1b-48e3-8aa1-02f5008fe44c" containerName="istio-proxy"
Oct 30 01:05:30.739740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.739729    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="489295d0-136f-47e6-9faf-4ebc614d6eb7" containerName="istio-proxy"
Oct 30 01:05:30.739834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.739824    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc6b4213-198c-4ba3-b56a-09fe015ae4d8" containerName="istio-proxy"
Oct 30 01:05:30.751625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.743999    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe8e2ee4-1059-413c-90d7-feaf02e70ed3" containerName="istio-proxy"
Oct 30 01:05:30.751820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.751801    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec1fb88b-9674-4709-aca3-445a57945d83" containerName="istio-proxy"
Oct 30 01:05:30.751954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.751941    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc09a1df-6c62-4f2a-937f-c68a4c95852c" containerName="istio-proxy"
Oct 30 01:05:30.805914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.805880    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:30.817410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.817376    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 01:05:30.817668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.817637    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 01:05:30.821105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.821068    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"]
Oct 30 01:05:30.823299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.823041    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d4622a\""
Oct 30 01:05:30.823462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.823440    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d4622a-dockercfg-r7km5\""
Oct 30 01:05:30.828943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.828921    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"]
Oct 30 01:05:30.833300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.833271    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:30.834091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.834072    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"]
Oct 30 01:05:30.972310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:30.972217    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:31.007758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.007726    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.007758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.007769    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8qq5\" (UniqueName: \"kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5\") pod \"payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.008049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.007896    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n22w\" (UniqueName: \"kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w\") pod \"payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.008113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.008059    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.110186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.110151    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.110700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.110302    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.110700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.110339    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8qq5\" (UniqueName: \"kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5\") pod \"payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.110700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.110439    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n22w\" (UniqueName: \"kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w\") pod \"payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.111823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.111795    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.112034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.111880    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.164193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.164161    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n22w\" (UniqueName: \"kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w\") pod \"payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.172974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.172944    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8qq5\" (UniqueName: \"kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5\") pod \"payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.183776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.183743    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:31.439977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.439670    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:31.509066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.504363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:31.635711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.635459    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:31.820879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.820221    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 01:05:31.820879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:31.820279    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 01:05:32.031638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.031608    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"]
Oct 30 01:05:32.032503 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:32.032477    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12af8977_9c14_4f55_806f_8753b42536e2.slice/crio-e9698cf64960f05cd626d5e8b688ef1c4f6a127d7ffe72e89c8ae2c90a123a2a WatchSource:0}: Error finding container e9698cf64960f05cd626d5e8b688ef1c4f6a127d7ffe72e89c8ae2c90a123a2a: Status 404 returned error can't find the container with id e9698cf64960f05cd626d5e8b688ef1c4f6a127d7ffe72e89c8ae2c90a123a2a
Oct 30 01:05:32.160899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.160864    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"]
Oct 30 01:05:32.161444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.161422    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:32.164011 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:32.163967    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cc2df4_8523_4fa6_ab4c_f7accbb3a4b5.slice/crio-b7e774f759a05f476ff5ffb88886033cd2ca901abeb58f228c87c61b8c005ef6 WatchSource:0}: Error finding container b7e774f759a05f476ff5ffb88886033cd2ca901abeb58f228c87c61b8c005ef6: Status 404 returned error can't find the container with id b7e774f759a05f476ff5ffb88886033cd2ca901abeb58f228c87c61b8c005ef6
Oct 30 01:05:32.849049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.847141    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:05:32.849049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.848639    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:05:32.849601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.849583    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:05:32.913423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.913383    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td" event={"ID":"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5","Type":"ContainerStarted","Data":"b7e774f759a05f476ff5ffb88886033cd2ca901abeb58f228c87c61b8c005ef6"}
Oct 30 01:05:32.916310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:32.916285    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h" event={"ID":"12af8977-9c14-4f55-806f-8753b42536e2","Type":"ContainerStarted","Data":"e9698cf64960f05cd626d5e8b688ef1c4f6a127d7ffe72e89c8ae2c90a123a2a"}
Oct 30 01:05:33.537279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:33.537209    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:34.941165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:34.941127    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td" event={"ID":"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5","Type":"ContainerStarted","Data":"2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3"}
Oct 30 01:05:34.941643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:34.941214    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:34.942823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:34.942801    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h" event={"ID":"12af8977-9c14-4f55-806f-8753b42536e2","Type":"ContainerStarted","Data":"1efccff75ff779cc0b1c8a6291b800c3f82f159528042c67cb184882349afa94"}
Oct 30 01:05:34.942995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:34.942957    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:34.989944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:34.989894    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td" podStartSLOduration=2.856372625 podStartE2EDuration="4.989880296s" totalImagesPullingTime="2.133507671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:34.986963974 +0000 UTC m=+1443.016274205" watchObservedRunningTime="2026-10-30 01:05:34.989880296 +0000 UTC m=+1443.019190523"
Oct 30 01:05:35.031566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:35.031517    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h" podStartSLOduration=2.80326901 podStartE2EDuration="5.031503747s" totalImagesPullingTime="2.228234737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:35.031306719 +0000 UTC m=+1443.060616949" watchObservedRunningTime="2026-10-30 01:05:35.031503747 +0000 UTC m=+1443.060813990"
Oct 30 01:05:35.327473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:35.327418    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:37.212169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:37.212138    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:37.343702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:37.343643    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:37.629605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:37.629561    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:38.579488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.579430    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:38.693634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.693605    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:38.844837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.844738    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"]
Oct 30 01:05:38.910582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.909953    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"]
Oct 30 01:05:38.919216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.918763    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"]
Oct 30 01:05:38.919216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.919013    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy" containerID="cri-o://9ee6b03ce406f3186f5e2bef2c886afb9b74fbfd7560e1d26953bc1aaa36a6d4" gracePeriod=30
Oct 30 01:05:38.971097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:38.971052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:39.168927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.123840    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:39.332903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.311444    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:39.386065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.386031    4577 scope.go:122] "RemoveContainer" containerID="bf2bd8ee651bd7b69b742d9a2130b4284082c61168c815fc89c95466723b65fc"
Oct 30 01:05:39.717219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.710815    4577 scope.go:122] "RemoveContainer" containerID="a99c541a43e06e34940f6698d30308d169002f08526b4185a4713c381c5b41c6"
Oct 30 01:05:39.799071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.799050    4577 scope.go:122] "RemoveContainer" containerID="33c96ed52a89b3933529b2e877789244d4f13725826d90cd527d5b174536715b"
Oct 30 01:05:39.813266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.813245    4577 scope.go:122] "RemoveContainer" containerID="44dcdd465ac6e1b4055e155097f4d30bceebd2a00c32de8f92640b82ad5c77d8"
Oct 30 01:05:39.824515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.824496    4577 scope.go:122] "RemoveContainer" containerID="a3d3f75cce93e486f54c0cee64b819443fc16fa596793801eda14e6dbb16efc8"
Oct 30 01:05:39.834461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.834441    4577 scope.go:122] "RemoveContainer" containerID="2a248348966459fa1ecce4618282aa3295c8f1dce4190cfe5250e1805f48efb7"
Oct 30 01:05:39.855647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.855624    4577 scope.go:122] "RemoveContainer" containerID="a3ab27521f2c41e240d622cb41f0e0610c8ac211b4c17972e4494e826df4723b"
Oct 30 01:05:39.869588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.869561    4577 scope.go:122] "RemoveContainer" containerID="2946467c91ce394084c65b7ea9cb462fb4fd1a9f4f2d17da7d33413aa1d9ec30"
Oct 30 01:05:39.880494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:39.880469    4577 scope.go:122] "RemoveContainer" containerID="f2e0f503ab8a24d3597c27e61b6ac49aab992d5a3bbcfcf3263df1c9d75ba621"
Oct 30 01:05:40.755581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:40.755548    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:42.342290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.342253    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"]
Oct 30 01:05:42.342950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.342516    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff" podUID="b3ed7daf-92a2-4bc9-a548-fbdac75692a2" containerName="maas-api" containerID="cri-o://6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183" gracePeriod=30
Oct 30 01:05:42.410751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.410696    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:42.864883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.859233    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:05:42.943884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.943252    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs\") pod \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") "
Oct 30 01:05:42.943884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.943349    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls\") pod \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") "
Oct 30 01:05:42.943884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.943435    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf\" (UniqueName: \"kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf\") pod \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") "
Oct 30 01:05:42.943884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.943523    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca\") pod \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\" (UID: \"b3ed7daf-92a2-4bc9-a548-fbdac75692a2\") "
Oct 30 01:05:42.948880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.944733    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca" pod "b3ed7daf-92a2-4bc9-a548-fbdac75692a2" (UID: "b3ed7daf-92a2-4bc9-a548-fbdac75692a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:43.005135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.957389    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf" pod "b3ed7daf-92a2-4bc9-a548-fbdac75692a2" (UID: "b3ed7daf-92a2-4bc9-a548-fbdac75692a2"). InnerVolumeSpecName "kube-api-access-cl7hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:43.005135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.988229    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls" pod "b3ed7daf-92a2-4bc9-a548-fbdac75692a2" (UID: "b3ed7daf-92a2-4bc9-a548-fbdac75692a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:05:43.005135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.993259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:43.005135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:42.995200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs" pod "b3ed7daf-92a2-4bc9-a548-fbdac75692a2" (UID: "b3ed7daf-92a2-4bc9-a548-fbdac75692a2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:05:43.052711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.052625    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:43.052711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.052672    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl7hf\" (UniqueName: \"kubernetes.io/projected/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-kube-api-access-cl7hf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:43.052711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.052683    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:43.052711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.052692    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3ed7daf-92a2-4bc9-a548-fbdac75692a2-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:43.085530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.085498    4577 generic.go:356] "Generic (PLEG): container finished" podID="b3ed7daf-92a2-4bc9-a548-fbdac75692a2" containerID="6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183" exitCode=0
Oct 30 01:05:43.085729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.085555    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"
Oct 30 01:05:43.085729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.085582    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff" event={"ID":"b3ed7daf-92a2-4bc9-a548-fbdac75692a2","Type":"ContainerDied","Data":"6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183"}
Oct 30 01:05:43.085729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.085638    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff" event={"ID":"b3ed7daf-92a2-4bc9-a548-fbdac75692a2","Type":"ContainerDied","Data":"5dc4a63e1a93584f2a1394a5aec26017c7c38767c4d068bc3a564bd39c377ca9"}
Oct 30 01:05:43.085729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.085663    4577 scope.go:122] "RemoveContainer" containerID="6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183"
Oct 30 01:05:43.098032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.098012    4577 scope.go:122] "RemoveContainer" containerID="6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183"
Oct 30 01:05:43.098330 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:43.098310    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183\": container with ID starting with 6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183 not found: ID does not exist" containerID="6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183"
Oct 30 01:05:43.098415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.098340    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183"} err="failed to get container status \"6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183\": rpc error: code = NotFound desc = could not find container \"6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183\": container with ID starting with 6fa2d251e637be7673a12f3c889cacbb3a478a6bc8b76bf829d3e20c32f8e183 not found: ID does not exist"
Oct 30 01:05:43.104918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.104898    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:43.146554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.146518    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"]
Oct 30 01:05:43.215455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.208734    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-1c78b8-654dcbfd75-h4nff"]
Oct 30 01:05:43.608701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.608666    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:43.845541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.845495    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7" podUID="dad28836-2b30-4803-9bb7-fa02c25075fa" containerName="payload-processing" containerID="cri-o://d4ad16ff1891b06cdfddc47b9215fd9d2a1b696e8fc18e17db5cfd6417a0a983" gracePeriod=25
Oct 30 01:05:43.910880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:43.910809    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk" podUID="932a7988-cdb7-47df-bbd5-29f2d0929804" containerName="payload-pre-processing" containerID="cri-o://daf87e4392ff0ba72dd5f13024a2d044a7b9fde34fc53131fe080b3a84e9097f" gracePeriod=25
Oct 30 01:05:44.102159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.102086    4577 generic.go:356] "Generic (PLEG): container finished" podID="e737e0f8-63b5-48e0-8350-6791d948044a" containerID="9ee6b03ce406f3186f5e2bef2c886afb9b74fbfd7560e1d26953bc1aaa36a6d4" exitCode=0
Oct 30 01:05:44.102309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.102164    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" event={"ID":"e737e0f8-63b5-48e0-8350-6791d948044a","Type":"ContainerDied","Data":"9ee6b03ce406f3186f5e2bef2c886afb9b74fbfd7560e1d26953bc1aaa36a6d4"}
Oct 30 01:05:44.102309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.102230    4577 scope.go:122] "RemoveContainer" containerID="5f9b5941720bf406379ad7db75821e9c4273242c3f5e46a9f37a782a95e2af26"
Oct 30 01:05:44.111682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.111649    4577 generic.go:356] "Generic (PLEG): container finished" podID="dad28836-2b30-4803-9bb7-fa02c25075fa" containerID="d4ad16ff1891b06cdfddc47b9215fd9d2a1b696e8fc18e17db5cfd6417a0a983" exitCode=0
Oct 30 01:05:44.114128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.113929    4577 generic.go:356] "Generic (PLEG): container finished" podID="932a7988-cdb7-47df-bbd5-29f2d0929804" containerID="daf87e4392ff0ba72dd5f13024a2d044a7b9fde34fc53131fe080b3a84e9097f" exitCode=0
Oct 30 01:05:44.117464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.117441    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3ed7daf-92a2-4bc9-a548-fbdac75692a2" path="/var/lib/kubelet/pods/b3ed7daf-92a2-4bc9-a548-fbdac75692a2/volumes"
Oct 30 01:05:44.117810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.117790    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7" event={"ID":"dad28836-2b30-4803-9bb7-fa02c25075fa","Type":"ContainerDied","Data":"d4ad16ff1891b06cdfddc47b9215fd9d2a1b696e8fc18e17db5cfd6417a0a983"}
Oct 30 01:05:44.117988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.117835    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk" event={"ID":"932a7988-cdb7-47df-bbd5-29f2d0929804","Type":"ContainerDied","Data":"daf87e4392ff0ba72dd5f13024a2d044a7b9fde34fc53131fe080b3a84e9097f"}
Oct 30 01:05:44.234146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.234112    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:44.309056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.309020    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:44.371833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.371755    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:05:44.385179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.385150    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8\" (UniqueName: \"kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8\") pod \"932a7988-cdb7-47df-bbd5-29f2d0929804\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") "
Oct 30 01:05:44.385392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.385246    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume\") pod \"932a7988-cdb7-47df-bbd5-29f2d0929804\" (UID: \"932a7988-cdb7-47df-bbd5-29f2d0929804\") "
Oct 30 01:05:44.385647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.385620    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume" pod "932a7988-cdb7-47df-bbd5-29f2d0929804" (UID: "932a7988-cdb7-47df-bbd5-29f2d0929804"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:44.386225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.386205    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/932a7988-cdb7-47df-bbd5-29f2d0929804-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.388673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.388642    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8" pod "932a7988-cdb7-47df-bbd5-29f2d0929804" (UID: "932a7988-cdb7-47df-bbd5-29f2d0929804"). InnerVolumeSpecName "kube-api-access-4rzh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:44.487444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487413    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.487444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487450    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.487714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487558    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.487714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487679    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.487835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487715    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.487835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487748    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.487835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487764    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:44.487835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.487774    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.488072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.488038    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume\") pod \"dad28836-2b30-4803-9bb7-fa02c25075fa\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") "
Oct 30 01:05:44.488129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.488089    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.488180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.488145    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l\" (UniqueName: \"kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l\") pod \"dad28836-2b30-4803-9bb7-fa02c25075fa\" (UID: \"dad28836-2b30-4803-9bb7-fa02c25075fa\") "
Oct 30 01:05:44.488268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.488231    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token\") pod \"e737e0f8-63b5-48e0-8350-6791d948044a\" (UID: \"e737e0f8-63b5-48e0-8350-6791d948044a\") "
Oct 30 01:05:44.488323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.488302    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.488531    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume" pod "dad28836-2b30-4803-9bb7-fa02c25075fa" (UID: "dad28836-2b30-4803-9bb7-fa02c25075fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.489149    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.489249    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.489274    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.489299    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dad28836-2b30-4803-9bb7-fa02c25075fa-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.489318    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rzh8\" (UniqueName: \"kubernetes.io/projected/932a7988-cdb7-47df-bbd5-29f2d0929804-kube-api-access-4rzh8\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.489820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.489744    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:44.490797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.490773    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:44.491998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.491973    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:05:44.491998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.491984    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:05:44.492158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.492094    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l" pod "dad28836-2b30-4803-9bb7-fa02c25075fa" (UID: "dad28836-2b30-4803-9bb7-fa02c25075fa"). InnerVolumeSpecName "kube-api-access-dt54l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:44.492749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.492720    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:44.493339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.493317    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g" pod "e737e0f8-63b5-48e0-8350-6791d948044a" (UID: "e737e0f8-63b5-48e0-8350-6791d948044a"). InnerVolumeSpecName "kube-api-access-ddb9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:44.591084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591042    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt54l\" (UniqueName: \"kubernetes.io/projected/dad28836-2b30-4803-9bb7-fa02c25075fa-kube-api-access-dt54l\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591080    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591093    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591105    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591116    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e737e0f8-63b5-48e0-8350-6791d948044a-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591128    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e737e0f8-63b5-48e0-8350-6791d948044a-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591138    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e737e0f8-63b5-48e0-8350-6791d948044a-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.591387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.591149    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddb9g\" (UniqueName: \"kubernetes.io/projected/e737e0f8-63b5-48e0-8350-6791d948044a-kube-api-access-ddb9g\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:05:44.905015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:44.904979    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:45.124778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.124749    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"
Oct 30 01:05:45.125008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.124785    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj" event={"ID":"e737e0f8-63b5-48e0-8350-6791d948044a","Type":"ContainerDied","Data":"dbc23c061ff89fd2a37df448bf5adfcf6ef8b536be466c213fc419cba8aaa587"}
Oct 30 01:05:45.125008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.124879    4577 scope.go:122] "RemoveContainer" containerID="9ee6b03ce406f3186f5e2bef2c886afb9b74fbfd7560e1d26953bc1aaa36a6d4"
Oct 30 01:05:45.126450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.126430    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"
Oct 30 01:05:45.126539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.126465    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7" event={"ID":"dad28836-2b30-4803-9bb7-fa02c25075fa","Type":"ContainerDied","Data":"43c4c826bc16a9255a27aaceceb6d8a345c7eefdd803d0e0793b60bf0c61fafe"}
Oct 30 01:05:45.128440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.128408    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"
Oct 30 01:05:45.128576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.128438    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk" event={"ID":"932a7988-cdb7-47df-bbd5-29f2d0929804","Type":"ContainerDied","Data":"2710b291a142ac1fc4691cadd764b31716005063c561fdd78fc36af655b01f86"}
Oct 30 01:05:45.148170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.148145    4577 scope.go:122] "RemoveContainer" containerID="d4ad16ff1891b06cdfddc47b9215fd9d2a1b696e8fc18e17db5cfd6417a0a983"
Oct 30 01:05:45.161904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.161882    4577 scope.go:122] "RemoveContainer" containerID="daf87e4392ff0ba72dd5f13024a2d044a7b9fde34fc53131fe080b3a84e9097f"
Oct 30 01:05:45.189106 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:45.189067    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode737e0f8_63b5_48e0_8350_6791d948044a.slice/crio-dbc23c061ff89fd2a37df448bf5adfcf6ef8b536be466c213fc419cba8aaa587\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad28836_2b30_4803_9bb7_fa02c25075fa.slice/crio-43c4c826bc16a9255a27aaceceb6d8a345c7eefdd803d0e0793b60bf0c61fafe\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad28836_2b30_4803_9bb7_fa02c25075fa.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932a7988_cdb7_47df_bbd5_29f2d0929804.slice/crio-2710b291a142ac1fc4691cadd764b31716005063c561fdd78fc36af655b01f86\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode737e0f8_63b5_48e0_8350_6791d948044a.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 01:05:45.200432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.200402    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"]
Oct 30 01:05:45.204396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.204370    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-5558cb5458-89sr7"]
Oct 30 01:05:45.244864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.244812    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"]
Oct 30 01:05:45.250505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.250480    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-645126-openshift-default-667d76b56f-q9bjj"]
Oct 30 01:05:45.283608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.283575    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"]
Oct 30 01:05:45.287087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.287060    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-11fa15-684c77bc9f-85rvk"]
Oct 30 01:05:45.955917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.955891    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:05:45.956365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:45.955963    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:05:46.111741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:46.111713    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="932a7988-cdb7-47df-bbd5-29f2d0929804" path="/var/lib/kubelet/pods/932a7988-cdb7-47df-bbd5-29f2d0929804/volumes"
Oct 30 01:05:46.112211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:46.112197    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dad28836-2b30-4803-9bb7-fa02c25075fa" path="/var/lib/kubelet/pods/dad28836-2b30-4803-9bb7-fa02c25075fa/volumes"
Oct 30 01:05:46.112623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:46.112611    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" path="/var/lib/kubelet/pods/e737e0f8-63b5-48e0-8350-6791d948044a/volumes"
Oct 30 01:05:46.737457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:46.737428    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:49.073446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:49.073409    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:49.946114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:49.946083    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:50.067319 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.067282    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:50.182136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.182101    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:50.299963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.299929    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:50.572512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.571667    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"]
Oct 30 01:05:50.573830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.573710    4577 cpu_manager.go:414] "containerMap: removing container" podUID="932a7988-cdb7-47df-bbd5-29f2d0929804" containerName="payload-pre-processing"
Oct 30 01:05:50.574050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574031    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="932a7988-cdb7-47df-bbd5-29f2d0929804" containerName="payload-pre-processing"
Oct 30 01:05:50.574244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574177    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy"
Oct 30 01:05:50.574328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574316    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy"
Oct 30 01:05:50.574419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574409    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy"
Oct 30 01:05:50.574493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574483    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy"
Oct 30 01:05:50.574589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574580    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b3ed7daf-92a2-4bc9-a548-fbdac75692a2" containerName="maas-api"
Oct 30 01:05:50.574659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574651    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3ed7daf-92a2-4bc9-a548-fbdac75692a2" containerName="maas-api"
Oct 30 01:05:50.574741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574730    4577 cpu_manager.go:414] "containerMap: removing container" podUID="dad28836-2b30-4803-9bb7-fa02c25075fa" containerName="payload-processing"
Oct 30 01:05:50.574811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.574802    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dad28836-2b30-4803-9bb7-fa02c25075fa" containerName="payload-processing"
Oct 30 01:05:50.575407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.575094    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3ed7daf-92a2-4bc9-a548-fbdac75692a2" containerName="maas-api"
Oct 30 01:05:50.575536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.575523    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="dad28836-2b30-4803-9bb7-fa02c25075fa" containerName="payload-processing"
Oct 30 01:05:50.575637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.575627    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="932a7988-cdb7-47df-bbd5-29f2d0929804" containerName="payload-pre-processing"
Oct 30 01:05:50.575736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.575725    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy"
Oct 30 01:05:50.575819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.575809    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e737e0f8-63b5-48e0-8350-6791d948044a" containerName="istio-proxy"
Oct 30 01:05:50.582411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.582388    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.599101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.599073    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.599328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.599303    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.599500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.599480    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tls4n\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.599864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.599814    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.600026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.600007    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.600182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.600166    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.600372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.600276    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.600556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.600491    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.600917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.600896    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.613360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.613338    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"]
Oct 30 01:05:50.616373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.616354    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-ad449e-openshift-default-dockercfg-vmdkf\""
Oct 30 01:05:50.643612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.643573    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.713937    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.713985    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714078    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714194    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714233    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714310    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714408    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tls4n\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714457    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.714496    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.715010    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.715266    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.715825    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.725827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.716472    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.740186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.740154    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.770000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.769378    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.770000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.769900    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.787021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.786982    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.797736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.797668    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tls4n\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n\") pod \"e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") " pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.929920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.926519    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:50.939458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:50.939359    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:51.255006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.254947    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:51.373640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.373371    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:51.389322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.389291    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"]
Oct 30 01:05:51.390305 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:51.390272    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72aee840_e2dd_4599_a7db_ab3ee0b2098a.slice/crio-8aecaeebd1d9a9e4ffbdc0e4efb11161a6cbfb77589f0d42aed2f252596ffdde WatchSource:0}: Error finding container 8aecaeebd1d9a9e4ffbdc0e4efb11161a6cbfb77589f0d42aed2f252596ffdde: Status 404 returned error can't find the container with id 8aecaeebd1d9a9e4ffbdc0e4efb11161a6cbfb77589f0d42aed2f252596ffdde
Oct 30 01:05:51.393255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.393224    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:51.393336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.393295    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:51.393336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.393326    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:51.728926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.727918    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"]
Oct 30 01:05:51.734031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.733999    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.761867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.758883    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-a8ab86-openshift-default-dockercfg-nm6zl\""
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768077    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768136    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768258    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768314    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768364    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768390    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768535    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768592    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.771156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.768647    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpf2c\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.774220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.774190    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"]
Oct 30 01:05:51.780817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.780786    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885012    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885077    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885193    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885246    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885302    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885325    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885456    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885511    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.885873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.885563    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpf2c\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.894878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.886597    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.925871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.922451    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.925871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.923158    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.925871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.923367    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.925871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.925261    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.933870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.931017    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.955196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.950162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.957714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.957643    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpf2c\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:51.973873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:51.972682    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token\") pod \"e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") " pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:52.085933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.077134    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:52.224077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.224027    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" event={"ID":"72aee840-e2dd-4599-a7db-ab3ee0b2098a","Type":"ContainerStarted","Data":"8a69303d37b73ccd7c4036a87512a4296d1a89dad626ed369bcb57b815aa6753"}
Oct 30 01:05:52.224294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.224109    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" event={"ID":"72aee840-e2dd-4599-a7db-ab3ee0b2098a","Type":"ContainerStarted","Data":"8aecaeebd1d9a9e4ffbdc0e4efb11161a6cbfb77589f0d42aed2f252596ffdde"}
Oct 30 01:05:52.264176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.264126    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:52.278612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.278550    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podStartSLOduration=2.278528247 podStartE2EDuration="2.278528247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:52.276028621 +0000 UTC m=+1460.305338860" watchObservedRunningTime="2026-10-30 01:05:52.278528247 +0000 UTC m=+1460.307838486"
Oct 30 01:05:52.466913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.466023    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"]
Oct 30 01:05:52.467639 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:05:52.467610    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10f3bf4_0a71_4aad_866f_4bf48958f6b7.slice/crio-938fd40a02170758d60696a597d0be729a73625ea532a7d5d49a474a5d71308f WatchSource:0}: Error finding container 938fd40a02170758d60696a597d0be729a73625ea532a7d5d49a474a5d71308f: Status 404 returned error can't find the container with id 938fd40a02170758d60696a597d0be729a73625ea532a7d5d49a474a5d71308f
Oct 30 01:05:52.498102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.498016    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:52.498497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.498459    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:52.498708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.498680    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:05:52.635330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.635295    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:52.807928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.807786    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:52.935993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.927610    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:52.945378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:52.944333    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:53.315904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.298408    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" event={"ID":"b10f3bf4-0a71-4aad-866f-4bf48958f6b7","Type":"ContainerStarted","Data":"37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"}
Oct 30 01:05:53.315904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.298472    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:53.315904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.298529    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:05:53.315904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.298556    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" event={"ID":"b10f3bf4-0a71-4aad-866f-4bf48958f6b7","Type":"ContainerStarted","Data":"938fd40a02170758d60696a597d0be729a73625ea532a7d5d49a474a5d71308f"}
Oct 30 01:05:53.329016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.328950    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"]
Oct 30 01:05:53.384914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.358980    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podStartSLOduration=2.358958199 podStartE2EDuration="2.358958199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:53.356321488 +0000 UTC m=+1461.385631720" watchObservedRunningTime="2026-10-30 01:05:53.358958199 +0000 UTC m=+1461.388268448"
Oct 30 01:05:53.384914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.379571    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"]
Oct 30 01:05:53.384914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.379703    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.400216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.400183    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-11fa15-dockercfg-bs5qt\""
Oct 30 01:05:53.400368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.400289    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-11fa15\""
Oct 30 01:05:53.490861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.490814    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vff4f\" (UniqueName: \"kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f\") pod \"payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.491076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.490912    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.637928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.636648    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vff4f\" (UniqueName: \"kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f\") pod \"payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.637928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.636709    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.639561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.639533    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.687050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.687014    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vff4f\" (UniqueName: \"kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f\") pod \"payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.705696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.705665    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:53.981736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:53.981699    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:54.086869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.086330    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:54.123871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.114201    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:54.170870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.152721    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"]
Oct 30 01:05:54.305936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.305886    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m" event={"ID":"274b1218-9bc3-4101-b090-60f03462c303","Type":"ContainerStarted","Data":"68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727"}
Oct 30 01:05:54.306152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.305964    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m" event={"ID":"274b1218-9bc3-4101-b090-60f03462c303","Type":"ContainerStarted","Data":"d84b0b52c40eeb36ac7f76d4d6f7496d6242c9951686e07bc0f62132acc5e6f6"}
Oct 30 01:05:54.306152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.305995    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:05:54.306455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.306434    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:54.307383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.307361    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:05:54.420147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.420085    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m" podStartSLOduration=1.420063064 podStartE2EDuration="1.420063064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:54.367172605 +0000 UTC m=+1462.396482840" watchObservedRunningTime="2026-10-30 01:05:54.420063064 +0000 UTC m=+1462.449373297"
Oct 30 01:05:54.678919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.676745    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:54.807552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:54.807519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:55.893961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:55.890472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:56.390153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:56.387801    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:56.517774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:56.517733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:57.038147 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:05:57.038113    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:05:58.163459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:58.163407    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:58.304164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:58.304121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:58.904870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:58.904672    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:59.924720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:59.924635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:05:59.927728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:59.927701    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"]
Oct 30 01:05:59.945435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:05:59.945407    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"]
Oct 30 01:06:00.056800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:00.056764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:00.191153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:00.184738    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:01.696017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:01.695976    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:03.469003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.468972    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:03.687671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.687579    4577 generic.go:356] "Generic (PLEG): container finished" podID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerID="3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2" exitCode=0
Oct 30 01:06:03.687671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.687653    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" event={"ID":"6adfe7e6-3179-4162-b94c-566f2c3ffc96","Type":"ContainerDied","Data":"3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"}
Oct 30 01:06:03.688263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.688245    4577 scope.go:122] "RemoveContainer" containerID="3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"
Oct 30 01:06:03.690236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.690202    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:03.690373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.690292    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:03.690373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:03.690341    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:04.706879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:04.704998    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" event={"ID":"6adfe7e6-3179-4162-b94c-566f2c3ffc96","Type":"ContainerStarted","Data":"c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d"}
Oct 30 01:06:04.930016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:04.929971    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr" podUID="c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" containerName="payload-processing" containerID="cri-o://7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12" gracePeriod=25
Oct 30 01:06:04.951960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:04.951918    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb" podUID="063f74ab-081d-4447-bcf2-f3cf744d7b1c" containerName="payload-pre-processing" containerID="cri-o://95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778" gracePeriod=25
Oct 30 01:06:05.320890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.317587    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:06:05.324203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.324177    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:06:05.388958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.388923    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8\" (UniqueName: \"kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8\") pod \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") "
Oct 30 01:06:05.389128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.389066    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume\") pod \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\" (UID: \"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5\") "
Oct 30 01:06:05.391405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.390726    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume" pod "c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" (UID: "c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:05.395377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.395345    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8" pod "c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" (UID: "c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5"). InnerVolumeSpecName "kube-api-access-jjms8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:05.401044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.400961    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:05.417205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.417187    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:06:05.493616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.492109    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q\" (UniqueName: \"kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q\") pod \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") "
Oct 30 01:06:05.493616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.492403    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume\") pod \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\" (UID: \"063f74ab-081d-4447-bcf2-f3cf744d7b1c\") "
Oct 30 01:06:05.493616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.493160    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjms8\" (UniqueName: \"kubernetes.io/projected/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-kube-api-access-jjms8\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:05.493616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.493183    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:05.495381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.495350    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume" pod "063f74ab-081d-4447-bcf2-f3cf744d7b1c" (UID: "063f74ab-081d-4447-bcf2-f3cf744d7b1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:05.513784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.510406    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q" pod "063f74ab-081d-4447-bcf2-f3cf744d7b1c" (UID: "063f74ab-081d-4447-bcf2-f3cf744d7b1c"). InnerVolumeSpecName "kube-api-access-26j9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:05.595154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.595064    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26j9q\" (UniqueName: \"kubernetes.io/projected/063f74ab-081d-4447-bcf2-f3cf744d7b1c-kube-api-access-26j9q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:05.595154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.595096    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/063f74ab-081d-4447-bcf2-f3cf744d7b1c-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:05.668343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.668255    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:06:05.668659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.668638    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 01:06:05.669115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.668812    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 01:06:05.715383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.715331    4577 generic.go:356] "Generic (PLEG): container finished" podID="063f74ab-081d-4447-bcf2-f3cf744d7b1c" containerID="95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778" exitCode=0
Oct 30 01:06:05.715383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.715375    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"
Oct 30 01:06:05.715879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.715409    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb" event={"ID":"063f74ab-081d-4447-bcf2-f3cf744d7b1c","Type":"ContainerDied","Data":"95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778"}
Oct 30 01:06:05.715879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.715474    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb" event={"ID":"063f74ab-081d-4447-bcf2-f3cf744d7b1c","Type":"ContainerDied","Data":"faf5f498c67bf76a77fbfb0c22f082b616ed760f4b363619f584d18973b6ed28"}
Oct 30 01:06:05.715879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.715502    4577 scope.go:122] "RemoveContainer" containerID="95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778"
Oct 30 01:06:05.717314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.717289    4577 generic.go:356] "Generic (PLEG): container finished" podID="c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" containerID="7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12" exitCode=0
Oct 30 01:06:05.717443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.717335    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"
Oct 30 01:06:05.717443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.717374    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr" event={"ID":"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5","Type":"ContainerDied","Data":"7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12"}
Oct 30 01:06:05.717443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.717431    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr" event={"ID":"c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5","Type":"ContainerDied","Data":"ca5baae5fd707dc54c0c0ac5daa7772983c844b0692f64a86736644152e23d98"}
Oct 30 01:06:05.728895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.728876    4577 scope.go:122] "RemoveContainer" containerID="95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778"
Oct 30 01:06:05.729193 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:05.729174    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778\": container with ID starting with 95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778 not found: ID does not exist" containerID="95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778"
Oct 30 01:06:05.729293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.729207    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778"} err="failed to get container status \"95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778\": rpc error: code = NotFound desc = could not find container \"95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778\": container with ID starting with 95805b7a5d3999e3ff82ae06ad641f596b7e8fe04ec7502814b37e3fb1dd6778 not found: ID does not exist"
Oct 30 01:06:05.729293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.729231    4577 scope.go:122] "RemoveContainer" containerID="7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12"
Oct 30 01:06:05.741297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.741259    4577 scope.go:122] "RemoveContainer" containerID="7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12"
Oct 30 01:06:05.741615 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:05.741586    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12\": container with ID starting with 7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12 not found: ID does not exist" containerID="7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12"
Oct 30 01:06:05.741697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.741627    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12"} err="failed to get container status \"7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12\": rpc error: code = NotFound desc = could not find container \"7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12\": container with ID starting with 7aeaf22fce4c0b896983e40fae2d4499f4e4ab6447197132558843cf46178b12 not found: ID does not exist"
Oct 30 01:06:05.788938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.788906    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"]
Oct 30 01:06:05.795396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.795363    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76f654-6686676c5b-tdrwr"]
Oct 30 01:06:05.826102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.826010    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:05.836284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.836199    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"]
Oct 30 01:06:05.842581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:05.842551    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76f654-648d86656c-s49qb"]
Oct 30 01:06:06.114699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:06.114670    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="063f74ab-081d-4447-bcf2-f3cf744d7b1c" path="/var/lib/kubelet/pods/063f74ab-081d-4447-bcf2-f3cf744d7b1c/volumes"
Oct 30 01:06:06.115153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:06.115133    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" path="/var/lib/kubelet/pods/c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5/volumes"
Oct 30 01:06:06.269873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:06.268163    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:06.381450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:06.381421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:06.660425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:06.660335    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 01:06:06.660722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:06.660692    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 01:06:07.642605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:07.642569    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:07.652871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:07.651236    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 01:06:07.652871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:07.651298    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 01:06:08.671871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:08.671268    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 01:06:08.671871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:08.671342    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 01:06:09.657872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.654302    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 01:06:09.657872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.654383    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 01:06:09.763944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.763903    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"]
Oct 30 01:06:09.764771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.764746    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" containerName="payload-processing"
Oct 30 01:06:09.764920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.764772    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" containerName="payload-processing"
Oct 30 01:06:09.764920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.764818    4577 cpu_manager.go:414] "containerMap: removing container" podUID="063f74ab-081d-4447-bcf2-f3cf744d7b1c" containerName="payload-pre-processing"
Oct 30 01:06:09.764920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.764828    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="063f74ab-081d-4447-bcf2-f3cf744d7b1c" containerName="payload-pre-processing"
Oct 30 01:06:09.765088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.765014    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1f3c030-7ed2-4b0e-b9e6-0f38cac84ef5" containerName="payload-processing"
Oct 30 01:06:09.765088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.765037    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="063f74ab-081d-4447-bcf2-f3cf744d7b1c" containerName="payload-pre-processing"
Oct 30 01:06:09.769086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.769062    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.805610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.805553    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-ad449e-serving-cert\""
Oct 30 01:06:09.808450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.808421    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.808634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.808537    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mt7q\" (UniqueName: \"kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.808634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.808561    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.808634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.808596    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.815014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.814430    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"]
Oct 30 01:06:09.864200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.862345    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:09.910466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.910383    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mt7q\" (UniqueName: \"kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.910466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.910420    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.910466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.910455    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.910764 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:09.910549    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-serving-cert: secret "maas-api-e2e-ipp-b-ad449e-serving-cert" not found
Oct 30 01:06:09.910764 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:09.910622    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls podName:06ef12f9-f0fb-4e0e-b131-7ae244af3855 nodeName:}" failed. No retries permitted until 2026-10-30 01:06:10.410603161 +0000 UTC m=+1478.439913393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls") pod "maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" (UID: "06ef12f9-f0fb-4e0e-b131-7ae244af3855") : secret "maas-api-e2e-ipp-b-ad449e-serving-cert" not found
Oct 30 01:06:09.910764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.910687    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.911405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.911379    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.914019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.913990    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.936946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.936910    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mt7q\" (UniqueName: \"kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:09.991894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:09.991809    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:10.112435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.112399    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:10.417715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.417684    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:10.420396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.420371    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:10.653831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.653799    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:06:10.654156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.654124    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:06:10.654793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.654775    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:06:10.710683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:10.710607    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:11.064235 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:06:11.059984    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ef12f9_f0fb_4e0e_b131_7ae244af3855.slice/crio-d38136ab6a1147ee4d5e09631b12e67b8d836605c68e64cd19496c6e1068e484 WatchSource:0}: Error finding container d38136ab6a1147ee4d5e09631b12e67b8d836605c68e64cd19496c6e1068e484: Status 404 returned error can't find the container with id d38136ab6a1147ee4d5e09631b12e67b8d836605c68e64cd19496c6e1068e484
Oct 30 01:06:11.070559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:11.070229    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"]
Oct 30 01:06:11.586014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:11.585976    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:11.836153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:11.836115    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" event={"ID":"06ef12f9-f0fb-4e0e-b131-7ae244af3855","Type":"ContainerStarted","Data":"7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"}
Oct 30 01:06:11.836671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:11.836216    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" event={"ID":"06ef12f9-f0fb-4e0e-b131-7ae244af3855","Type":"ContainerStarted","Data":"d38136ab6a1147ee4d5e09631b12e67b8d836605c68e64cd19496c6e1068e484"}
Oct 30 01:06:11.836671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:11.836252    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:11.885218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:11.885159    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" podStartSLOduration=2.477523173 podStartE2EDuration="2.885141211s" totalImagesPullingTime="407.618038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:11.881646582 +0000 UTC m=+1479.910956814" watchObservedRunningTime="2026-10-30 01:06:11.885141211 +0000 UTC m=+1479.914451443"
Oct 30 01:06:13.294656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.294578    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"]
Oct 30 01:06:13.298076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.298056    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.320043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.320012    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"]
Oct 30 01:06:13.336231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.327532    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a8ab86-serving-cert\""
Oct 30 01:06:13.350670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.350634    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.350968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.350697    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.350968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.350773    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.350968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.350874    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pctbn\" (UniqueName: \"kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.398388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.389246    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:13.452069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.452030    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pctbn\" (UniqueName: \"kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.452280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.452238    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.452351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.452303    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.452404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.452351    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.453040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.453014    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.455076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.455055    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.455076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.455068    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.477352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.477327    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pctbn\" (UniqueName: \"kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn\") pod \"maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.508616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.508581    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:13.612922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.612420    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:13.661137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:13.661107    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:14.004137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:14.004103    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"]
Oct 30 01:06:14.004408 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:06:14.004382    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dbf5831_1044_4ef5_92c4_940cb0d3cdd1.slice/crio-93e082fc1f0d854f9bbf0d7f0bd20c1dd2567759ef62c63e52558cf3d8cf4b53 WatchSource:0}: Error finding container 93e082fc1f0d854f9bbf0d7f0bd20c1dd2567759ef62c63e52558cf3d8cf4b53: Status 404 returned error can't find the container with id 93e082fc1f0d854f9bbf0d7f0bd20c1dd2567759ef62c63e52558cf3d8cf4b53
Oct 30 01:06:14.864625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:14.864566    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45" event={"ID":"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1","Type":"ContainerStarted","Data":"c8d2b68ae72d061403fc903dc808c54f5fd73593784aa0fba67b7391b4f4b518"}
Oct 30 01:06:14.865115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:14.864641    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45" event={"ID":"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1","Type":"ContainerStarted","Data":"93e082fc1f0d854f9bbf0d7f0bd20c1dd2567759ef62c63e52558cf3d8cf4b53"}
Oct 30 01:06:14.865115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:14.864776    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:14.913027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:14.912967    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45" podStartSLOduration=1.5567325379999999 podStartE2EDuration="1.912946573s" totalImagesPullingTime="356.214035ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:14.909625089 +0000 UTC m=+1482.938935525" watchObservedRunningTime="2026-10-30 01:06:14.912946573 +0000 UTC m=+1482.942256804"
Oct 30 01:06:15.205837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:15.205757    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:16.904558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:16.904520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:17.846960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:17.846910    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.147:8443/health\": dial tcp 10.134.0.147:8443: connect: connection refused"
Oct 30 01:06:18.697655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:18.697621    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:20.457330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:20.457274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:20.878215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:20.878187    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:06:20.940712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:20.940671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:21.062381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:21.062342    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:21.924392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:21.924291    4577 generic.go:356] "Generic (PLEG): container finished" podID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerID="7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357" exitCode=1
Oct 30 01:06:21.924392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:21.924368    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" event={"ID":"06ef12f9-f0fb-4e0e-b131-7ae244af3855","Type":"ContainerDied","Data":"7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"}
Oct 30 01:06:21.924986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:21.924970    4577 scope.go:122] "RemoveContainer" containerID="7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"
Oct 30 01:06:22.214224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:22.214095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:22.933926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:22.933892    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" event={"ID":"06ef12f9-f0fb-4e0e-b131-7ae244af3855","Type":"ContainerStarted","Data":"6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099"}
Oct 30 01:06:22.934312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:22.934096    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:24.431636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:24.431599    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:25.925818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:25.925785    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"]
Oct 30 01:06:25.926346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:25.926013    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p" podUID="74d9d96f-59af-4a77-b8d3-71b3309cf224" containerName="maas-api" containerID="cri-o://f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926" gracePeriod=30
Oct 30 01:06:25.989505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:25.989469    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:26.640919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:26.617044    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:26.786125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:26.785119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:26.895001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:26.894427    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:06:27.002532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.002502    4577 generic.go:356] "Generic (PLEG): container finished" podID="74d9d96f-59af-4a77-b8d3-71b3309cf224" containerID="f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926" exitCode=0
Oct 30 01:06:27.003008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.002557    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"
Oct 30 01:06:27.003008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.002722    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p" event={"ID":"74d9d96f-59af-4a77-b8d3-71b3309cf224","Type":"ContainerDied","Data":"f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926"}
Oct 30 01:06:27.003008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.002758    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p" event={"ID":"74d9d96f-59af-4a77-b8d3-71b3309cf224","Type":"ContainerDied","Data":"97c6ae2f88212ed399b1e77e8ddecaf2bbb7a07e2c35f3218f58a4be83b44965"}
Oct 30 01:06:27.003008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.002778    4577 scope.go:122] "RemoveContainer" containerID="f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926"
Oct 30 01:06:27.014510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.014485    4577 scope.go:122] "RemoveContainer" containerID="f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926"
Oct 30 01:06:27.015071 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:27.015028    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926\": container with ID starting with f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926 not found: ID does not exist" containerID="f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926"
Oct 30 01:06:27.015170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.015079    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926"} err="failed to get container status \"f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926\": rpc error: code = NotFound desc = could not find container \"f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926\": container with ID starting with f867c46df8dd14107e6f8d539d8e177156e6ba3920ef97d45c2f9f865fd9d926 not found: ID does not exist"
Oct 30 01:06:27.017477 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:27.017450    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:06:27.044501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.044476    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca\") pod \"74d9d96f-59af-4a77-b8d3-71b3309cf224\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") "
Oct 30 01:06:27.044644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.044562    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk\" (UniqueName: \"kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk\") pod \"74d9d96f-59af-4a77-b8d3-71b3309cf224\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") "
Oct 30 01:06:27.044644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.044602    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls\") pod \"74d9d96f-59af-4a77-b8d3-71b3309cf224\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") "
Oct 30 01:06:27.044767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.044690    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs\") pod \"74d9d96f-59af-4a77-b8d3-71b3309cf224\" (UID: \"74d9d96f-59af-4a77-b8d3-71b3309cf224\") "
Oct 30 01:06:27.062582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.044899    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca" pod "74d9d96f-59af-4a77-b8d3-71b3309cf224" (UID: "74d9d96f-59af-4a77-b8d3-71b3309cf224"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:27.062582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.045620    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d9d96f-59af-4a77-b8d3-71b3309cf224-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:27.100232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.099000    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs" pod "74d9d96f-59af-4a77-b8d3-71b3309cf224" (UID: "74d9d96f-59af-4a77-b8d3-71b3309cf224"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:06:27.100232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.099134    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk" pod "74d9d96f-59af-4a77-b8d3-71b3309cf224" (UID: "74d9d96f-59af-4a77-b8d3-71b3309cf224"). InnerVolumeSpecName "kube-api-access-zxmnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:27.100232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.099951    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls" pod "74d9d96f-59af-4a77-b8d3-71b3309cf224" (UID: "74d9d96f-59af-4a77-b8d3-71b3309cf224"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:06:27.169957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.160029    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxmnk\" (UniqueName: \"kubernetes.io/projected/74d9d96f-59af-4a77-b8d3-71b3309cf224-kube-api-access-zxmnk\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:27.169957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.160062    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:27.169957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.160078    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74d9d96f-59af-4a77-b8d3-71b3309cf224-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:27.169957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.165971    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:27.373884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.370111    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"]
Oct 30 01:06:27.383323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:27.383286    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-76f654-6646b8b9c4-s6q8p"]
Oct 30 01:06:28.119500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:28.119416    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74d9d96f-59af-4a77-b8d3-71b3309cf224" path="/var/lib/kubelet/pods/74d9d96f-59af-4a77-b8d3-71b3309cf224/volumes"
Oct 30 01:06:28.946902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:28.946873    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:06:29.024181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:29.024142    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:30.829430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:30.829376    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:31.410396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.410358    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"]
Oct 30 01:06:31.411547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.411522    4577 cpu_manager.go:414] "containerMap: removing container" podUID="74d9d96f-59af-4a77-b8d3-71b3309cf224" containerName="maas-api"
Oct 30 01:06:31.411547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.411548    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74d9d96f-59af-4a77-b8d3-71b3309cf224" containerName="maas-api"
Oct 30 01:06:31.411760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.411748    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="74d9d96f-59af-4a77-b8d3-71b3309cf224" containerName="maas-api"
Oct 30 01:06:31.431876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.419619    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.435726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.435567    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"]
Oct 30 01:06:31.443113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.443080    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-838fd5-openshift-default-dockercfg-2rtt5\""
Oct 30 01:06:31.464316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.464280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:31.489824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.489796    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490062    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490103    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k567k\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490243    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490303    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490349    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490393    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490463    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.490534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.490496    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.600864    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.600935    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.600984    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.601058    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.601088    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.601125    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.601314    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.601347    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k567k\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.601450    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.602092    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.602362    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.603876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.603052    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.614883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.604991    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.614883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.605596    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.614883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.608690    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:31.650562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.629101    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.681915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.681714    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.684050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.682959    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.704928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.696621    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k567k\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k\") pod \"e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") " pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:31.748183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:31.748142    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:32.190722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:32.190692    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"]
Oct 30 01:06:32.196437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:32.196402    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:32.196554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:32.196470    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:32.196554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:32.196498    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:32.814859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:32.814793    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:33.065188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.065151    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" event={"ID":"47dcdb1c-2440-4ffc-aecd-8f7dae131d12","Type":"ContainerStarted","Data":"6a2a86556eba28fa9e9b81775485d9b45ec369a01869bca446973977784aa3ff"}
Oct 30 01:06:33.065408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.065258    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" event={"ID":"47dcdb1c-2440-4ffc-aecd-8f7dae131d12","Type":"ContainerStarted","Data":"d07e4221aed1705189f3e8dc20470a26ad72c6859e9764d97d41678308940e91"}
Oct 30 01:06:33.067426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.067402    4577 generic.go:356] "Generic (PLEG): container finished" podID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerID="8a69303d37b73ccd7c4036a87512a4296d1a89dad626ed369bcb57b815aa6753" exitCode=0
Oct 30 01:06:33.067571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.067431    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" event={"ID":"72aee840-e2dd-4599-a7db-ab3ee0b2098a","Type":"ContainerDied","Data":"8a69303d37b73ccd7c4036a87512a4296d1a89dad626ed369bcb57b815aa6753"}
Oct 30 01:06:33.067803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.067789    4577 scope.go:122] "RemoveContainer" containerID="8a69303d37b73ccd7c4036a87512a4296d1a89dad626ed369bcb57b815aa6753"
Oct 30 01:06:33.069388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.069359    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:33.069508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.069431    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:33.069508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.069462    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:06:33.130456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.130409    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podStartSLOduration=2.130390565 podStartE2EDuration="2.130390565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:33.127368065 +0000 UTC m=+1501.156678317" watchObservedRunningTime="2026-10-30 01:06:33.130390565 +0000 UTC m=+1501.159700797"
Oct 30 01:06:33.750981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.750948    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:33.761200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.761165    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:33.817339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.817309    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:33.933822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:33.933782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:34.080964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.080875    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" event={"ID":"72aee840-e2dd-4599-a7db-ab3ee0b2098a","Type":"ContainerStarted","Data":"2e6606805ecc4bcc881c4df1b83b022e139b9d4d27a918b7e3e77b9401b66a36"}
Oct 30 01:06:34.081328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.081307    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:34.082259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.082237    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:06:34.190216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.190175    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:34.205702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.205674    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"]
Oct 30 01:06:34.208167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.208124    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy" containerID="cri-o://9df1a3a8bd3c8d07f81316f28b07045e4c7d6187e67633706a1eb7d1236392c9" gracePeriod=30
Oct 30 01:06:34.210492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.210473    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"]
Oct 30 01:06:34.210683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.210664    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy" containerID="cri-o://e0dbbee626eae7d03c9df408ab074aef3a73885dc77bd2debc267af5647c7d91" gracePeriod=30
Oct 30 01:06:34.686053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.686000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:34.927257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.927224    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:06:34.928263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.928241    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:06:34.928325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:34.928301    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:06:35.120311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:35.120276    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:35.520493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:35.520423    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:35.660869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:35.660408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:35.912926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:35.903674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:35.992871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:35.942558    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:06:35.992871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:35.942640    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:06:36.031951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:36.031913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:36.720408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:36.718838    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:36.843979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:36.843951    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:36.928925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:36.928191    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:06:36.928925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:36.928255    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:06:36.996595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:36.996464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:37.777529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.777487    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:37.906286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.906251    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:37.915992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.915957    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"]
Oct 30 01:06:37.921836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.921801    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:37.928077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.928047    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:06:37.928418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.928227    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:06:37.939308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.939283    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"]
Oct 30 01:06:38.001873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:37.989149    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-838fd5\""
Oct 30 01:06:38.026327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.000173    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-838fd5-dockercfg-4qjbv\""
Oct 30 01:06:38.026601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.005122    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.061982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.053414    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwc29\" (UniqueName: \"kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29\") pod \"payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.259663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.259630    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.261624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.261561    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.262161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.262141    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwc29\" (UniqueName: \"kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29\") pod \"payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.299373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.299332    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:06:38.301401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.300244    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwc29\" (UniqueName: \"kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29\") pod \"payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.567872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.566126    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:38.673902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.673817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:38.887813 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:06:38.887774    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c9ea7a_bb62_4353_ae53_0b0c3d887895.slice/crio-ada0514401f8513454e9fc65b107384b8546522039c3b2810e56309832c2bb58 WatchSource:0}: Error finding container ada0514401f8513454e9fc65b107384b8546522039c3b2810e56309832c2bb58: Status 404 returned error can't find the container with id ada0514401f8513454e9fc65b107384b8546522039c3b2810e56309832c2bb58
Oct 30 01:06:38.889731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.889704    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"]
Oct 30 01:06:38.928668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.928636    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:06:38.928831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:38.928696    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:06:39.361878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.345241    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc" event={"ID":"e5c9ea7a-bb62-4353-ae53-0b0c3d887895","Type":"ContainerStarted","Data":"9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07"}
Oct 30 01:06:39.361878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.345317    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc" event={"ID":"e5c9ea7a-bb62-4353-ae53-0b0c3d887895","Type":"ContainerStarted","Data":"ada0514401f8513454e9fc65b107384b8546522039c3b2810e56309832c2bb58"}
Oct 30 01:06:39.361878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.358558    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:39.380873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.379908    4577 generic.go:356] "Generic (PLEG): container finished" podID="765e057d-2b90-442f-bec5-78841a2d9619" containerID="9df1a3a8bd3c8d07f81316f28b07045e4c7d6187e67633706a1eb7d1236392c9" exitCode=0
Oct 30 01:06:39.380873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.380036    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" event={"ID":"765e057d-2b90-442f-bec5-78841a2d9619","Type":"ContainerDied","Data":"9df1a3a8bd3c8d07f81316f28b07045e4c7d6187e67633706a1eb7d1236392c9"}
Oct 30 01:06:39.380873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.380113    4577 scope.go:122] "RemoveContainer" containerID="421e190c7c7910cb2d8389f448893e5260779a1c21b9826fcea0f3d883a93b4d"
Oct 30 01:06:39.382696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.382665    4577 generic.go:356] "Generic (PLEG): container finished" podID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerID="e0dbbee626eae7d03c9df408ab074aef3a73885dc77bd2debc267af5647c7d91" exitCode=0
Oct 30 01:06:39.382902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.382740    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" event={"ID":"a75c5f9f-7df6-4350-907d-f4b412ed4318","Type":"ContainerDied","Data":"e0dbbee626eae7d03c9df408ab074aef3a73885dc77bd2debc267af5647c7d91"}
Oct 30 01:06:39.405598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.405523    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc" podStartSLOduration=2.405509324 podStartE2EDuration="2.405509324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:39.403813936 +0000 UTC m=+1507.433124168" watchObservedRunningTime="2026-10-30 01:06:39.405509324 +0000 UTC m=+1507.434819555"
Oct 30 01:06:39.733899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.733816    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:06:39.736000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.735980    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:06:39.818403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.818360    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.818570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.818512    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.818657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.818628    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.818721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.818699    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.819070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819047    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.819218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819152    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.819286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819234    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.819354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819325    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.819414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819386    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.819532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819518    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.819602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819551    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.819795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819778    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.820000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.819982    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.820227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.820210    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:39.820557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.820539    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.820910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.820892    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.821072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821056    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.821308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821291    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:39.821386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821303    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.821472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821457    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.821537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821502    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.821600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821544    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.821654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821598    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.821723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821709    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token\") pod \"a75c5f9f-7df6-4350-907d-f4b412ed4318\" (UID: \"a75c5f9f-7df6-4350-907d-f4b412ed4318\") "
Oct 30 01:06:39.821779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.821742    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data\") pod \"765e057d-2b90-442f-bec5-78841a2d9619\" (UID: \"765e057d-2b90-442f-bec5-78841a2d9619\") "
Oct 30 01:06:39.822273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822256    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75c5f9f-7df6-4350-907d-f4b412ed4318-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822276    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822291    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822304    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765e057d-2b90-442f-bec5-78841a2d9619-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822316    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822328    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822341    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.822755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822735    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.822987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.822931    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.823195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.823163    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:39.823347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.823290    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.823469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.823452    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:06:39.823537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.823514    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.824207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.824173    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "kube-api-access-4qjpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:39.824413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.824393    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:39.825248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.825221    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "kube-api-access-9r9kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:39.825531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.825514    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo" pod "765e057d-2b90-442f-bec5-78841a2d9619" (UID: "765e057d-2b90-442f-bec5-78841a2d9619"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:06:39.826178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.826162    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token" pod "a75c5f9f-7df6-4350-907d-f4b412ed4318" (UID: "a75c5f9f-7df6-4350-907d-f4b412ed4318"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:39.923635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923596    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923632    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qjpp\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-kube-api-access-4qjpp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923643    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923651    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923659    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r9kw\" (UniqueName: \"kubernetes.io/projected/765e057d-2b90-442f-bec5-78841a2d9619-kube-api-access-9r9kw\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923667    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765e057d-2b90-442f-bec5-78841a2d9619-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923675    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923684    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923691    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75c5f9f-7df6-4350-907d-f4b412ed4318-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923698    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.923949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.923705    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765e057d-2b90-442f-bec5-78841a2d9619-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:39.928190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.928165    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:06:39.928301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:39.928211    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:06:40.039364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.039261    4577 scope.go:122] "RemoveContainer" containerID="ef2afc0474dd4d7827bd417e5e6fc904c79ef5baba446c7833835f7d1ef62486"
Oct 30 01:06:40.393811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.393775    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"
Oct 30 01:06:40.393811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.393793    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk" event={"ID":"765e057d-2b90-442f-bec5-78841a2d9619","Type":"ContainerDied","Data":"841fc51a2ae4f9e94a3c6cf9b8e15724a4142c9dba5e81edf1f4fedfdcaf742c"}
Oct 30 01:06:40.394369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.393892    4577 scope.go:122] "RemoveContainer" containerID="9df1a3a8bd3c8d07f81316f28b07045e4c7d6187e67633706a1eb7d1236392c9"
Oct 30 01:06:40.396542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.396509    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg" event={"ID":"a75c5f9f-7df6-4350-907d-f4b412ed4318","Type":"ContainerDied","Data":"d30d4a18299596d50179a51b9c81b55c8d1b306c6ff6215f6fae955a62528656"}
Oct 30 01:06:40.396665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.396601    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"
Oct 30 01:06:40.405202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.405180    4577 scope.go:122] "RemoveContainer" containerID="e0dbbee626eae7d03c9df408ab074aef3a73885dc77bd2debc267af5647c7d91"
Oct 30 01:06:40.428489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.428457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:40.456058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.456026    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"]
Oct 30 01:06:40.544162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.544119    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-dvcjk"]
Oct 30 01:06:40.554786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.554752    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"]
Oct 30 01:06:40.560711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.560681    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-76f654-openshift-default-cf5f45b55-wvghg"]
Oct 30 01:06:40.931572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.931544    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:06:40.932554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:40.932532    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:06:42.112793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.112760    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="765e057d-2b90-442f-bec5-78841a2d9619" path="/var/lib/kubelet/pods/765e057d-2b90-442f-bec5-78841a2d9619/volumes"
Oct 30 01:06:42.113344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.113328    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" path="/var/lib/kubelet/pods/a75c5f9f-7df6-4350-907d-f4b412ed4318/volumes"
Oct 30 01:06:42.165318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.165284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:42.328164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.328130    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"]
Oct 30 01:06:42.328370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.328348    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy" containerID="cri-o://b7ce996bcf2441757a481e79c6735e4b6cf489fad26a9ffeef2fe1bb5dd1a90e" gracePeriod=30
Oct 30 01:06:42.343303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.343269    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"]
Oct 30 01:06:42.343545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.343516    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy" containerID="cri-o://996675fcd1196c9e0d2e7dc9ab764743b16887757cde8524797108d43fcdb62f" gracePeriod=30
Oct 30 01:06:42.348420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.348390    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"]
Oct 30 01:06:42.348645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:42.348620    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy" containerID="cri-o://b216588f397d60972b5384bec8afab494643e0e8b4b44e8f793c8a7913e5bd34" gracePeriod=30
Oct 30 01:06:44.375595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:44.375560    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:46.154752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:46.154704    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:47.513862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.513247    4577 generic.go:356] "Generic (PLEG): container finished" podID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerID="b216588f397d60972b5384bec8afab494643e0e8b4b44e8f793c8a7913e5bd34" exitCode=0
Oct 30 01:06:47.513862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.513501    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" event={"ID":"71d6f8ed-df9e-4e61-951d-c4490161d7ae","Type":"ContainerDied","Data":"b216588f397d60972b5384bec8afab494643e0e8b4b44e8f793c8a7913e5bd34"}
Oct 30 01:06:47.518606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.515586    4577 generic.go:356] "Generic (PLEG): container finished" podID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerID="b7ce996bcf2441757a481e79c6735e4b6cf489fad26a9ffeef2fe1bb5dd1a90e" exitCode=0
Oct 30 01:06:47.518606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.515785    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" event={"ID":"f83157ed-e10d-4f90-a3bc-bec67e35b3b8","Type":"ContainerDied","Data":"b7ce996bcf2441757a481e79c6735e4b6cf489fad26a9ffeef2fe1bb5dd1a90e"}
Oct 30 01:06:47.518606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.517991    4577 generic.go:356] "Generic (PLEG): container finished" podID="926056e3-02b6-492f-8a1a-e93d5402b654" containerID="996675fcd1196c9e0d2e7dc9ab764743b16887757cde8524797108d43fcdb62f" exitCode=0
Oct 30 01:06:47.518606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.518065    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" event={"ID":"926056e3-02b6-492f-8a1a-e93d5402b654","Type":"ContainerDied","Data":"996675fcd1196c9e0d2e7dc9ab764743b16887757cde8524797108d43fcdb62f"}
Oct 30 01:06:47.723696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.716979    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:06:47.762941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.762912    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 01:06:47.766720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.766698    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:06:47.891361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891326    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.891555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891398    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.891555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891436    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.891555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891468    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.891555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891499    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.891788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891557    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.891788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891590    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.891788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891596    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.891788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891628    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.892067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891913    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.892067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.891928    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892301    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892344    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892356    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892389    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892466    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892495    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892528    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892564    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892594    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892671    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892728    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892756    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892802    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892836    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892948    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.892997    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.893004    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.893044    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert\") pod \"926056e3-02b6-492f-8a1a-e93d5402b654\" (UID: \"926056e3-02b6-492f-8a1a-e93d5402b654\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.893121    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.893153    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data\") pod \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\" (UID: \"71d6f8ed-df9e-4e61-951d-c4490161d7ae\") "
Oct 30 01:06:47.894433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.893216    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket\") pod \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\" (UID: \"f83157ed-e10d-4f90-a3bc-bec67e35b3b8\") "
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.894933    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895149    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895439    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895458    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895473    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895487    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895500    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895515    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.895659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895528    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:47.896104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.895895    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "kube-api-access-vz7c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:47.896146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.896118    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.896762    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897092    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897332    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897395    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "kube-api-access-45lwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897625    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897713    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897742    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.897919    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.898016    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.898130    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.898487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.898224    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.900403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.900370    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:47.900648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.900610    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.900648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.900623    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:47.900904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.900875    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo" pod "71d6f8ed-df9e-4e61-951d-c4490161d7ae" (UID: "71d6f8ed-df9e-4e61-951d-c4490161d7ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:06:47.903086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.903022    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "kube-api-access-4fqcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:47.903184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.903087    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo" pod "926056e3-02b6-492f-8a1a-e93d5402b654" (UID: "926056e3-02b6-492f-8a1a-e93d5402b654"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:06:47.906058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.905909    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo" pod "f83157ed-e10d-4f90-a3bc-bec67e35b3b8" (UID: "f83157ed-e10d-4f90-a3bc-bec67e35b3b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998252    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998289    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998302    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998314    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998326    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926056e3-02b6-492f-8a1a-e93d5402b654-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998339    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998350    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998363    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998376    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998388    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz7c6\" (UniqueName: \"kubernetes.io/projected/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-kube-api-access-vz7c6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998403    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998416    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998428    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998441    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45lwf\" (UniqueName: \"kubernetes.io/projected/71d6f8ed-df9e-4e61-951d-c4490161d7ae-kube-api-access-45lwf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998454    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fqcc\" (UniqueName: \"kubernetes.io/projected/926056e3-02b6-492f-8a1a-e93d5402b654-kube-api-access-4fqcc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998467    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926056e3-02b6-492f-8a1a-e93d5402b654-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998479    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926056e3-02b6-492f-8a1a-e93d5402b654-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998492    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998505    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d6f8ed-df9e-4e61-951d-c4490161d7ae-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.016885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:47.998548    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f83157ed-e10d-4f90-a3bc-bec67e35b3b8-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:06:48.527452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.527422    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"
Oct 30 01:06:48.527900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.527453    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk" event={"ID":"f83157ed-e10d-4f90-a3bc-bec67e35b3b8","Type":"ContainerDied","Data":"7a77e4f10f854e585e95e42d1ab36903d671b42b5b1ae2bf358ad9f5f5b40850"}
Oct 30 01:06:48.527900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.527512    4577 scope.go:122] "RemoveContainer" containerID="b7ce996bcf2441757a481e79c6735e4b6cf489fad26a9ffeef2fe1bb5dd1a90e"
Oct 30 01:06:48.530141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.529438    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"
Oct 30 01:06:48.530141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.529456    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8" event={"ID":"926056e3-02b6-492f-8a1a-e93d5402b654","Type":"ContainerDied","Data":"db987216a33e70b4a26f18ccad51ed384e142e268193b8797f9e68cd88e6a492"}
Oct 30 01:06:48.531002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.530978    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb" event={"ID":"71d6f8ed-df9e-4e61-951d-c4490161d7ae","Type":"ContainerDied","Data":"3d8d2e803b3b1ee50c59542fac8617acc8620ef90a59c9c3bfe225c22e8bf60d"}
Oct 30 01:06:48.531079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.531069    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"
Oct 30 01:06:48.544744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.544722    4577 scope.go:122] "RemoveContainer" containerID="996675fcd1196c9e0d2e7dc9ab764743b16887757cde8524797108d43fcdb62f"
Oct 30 01:06:48.555074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.555058    4577 scope.go:122] "RemoveContainer" containerID="b216588f397d60972b5384bec8afab494643e0e8b4b44e8f793c8a7913e5bd34"
Oct 30 01:06:48.556337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.556313    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:48.590278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.590256    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"]
Oct 30 01:06:48.656327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.656289    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-4w9vb"]
Oct 30 01:06:48.663506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.663463    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"]
Oct 30 01:06:48.668187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.668157    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-bs9wk"]
Oct 30 01:06:48.699282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.699253    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"]
Oct 30 01:06:48.702689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:48.702665    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-zldq8"]
Oct 30 01:06:49.914160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:49.914130    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:50.085203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.085167    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:50.112946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.112913    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" path="/var/lib/kubelet/pods/71d6f8ed-df9e-4e61-951d-c4490161d7ae/volumes"
Oct 30 01:06:50.113509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.113494    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" path="/var/lib/kubelet/pods/926056e3-02b6-492f-8a1a-e93d5402b654/volumes"
Oct 30 01:06:50.114007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.113994    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" path="/var/lib/kubelet/pods/f83157ed-e10d-4f90-a3bc-bec67e35b3b8/volumes"
Oct 30 01:06:50.317737 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.317687    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"]
Oct 30 01:06:50.319033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319005    4577 cpu_manager.go:414] "containerMap: removing container" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy"
Oct 30 01:06:50.319197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319183    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy"
Oct 30 01:06:50.319330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319318    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy"
Oct 30 01:06:50.319446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319416    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy"
Oct 30 01:06:50.319600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319587    4577 cpu_manager.go:414] "containerMap: removing container" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy"
Oct 30 01:06:50.319689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319677    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy"
Oct 30 01:06:50.319766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319756    4577 cpu_manager.go:414] "containerMap: removing container" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy"
Oct 30 01:06:50.319950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.319933    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy"
Oct 30 01:06:50.320070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320060    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy"
Oct 30 01:06:50.320152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320139    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy"
Oct 30 01:06:50.320475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320459    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a75c5f9f-7df6-4350-907d-f4b412ed4318" containerName="istio-proxy"
Oct 30 01:06:50.320583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320573    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="926056e3-02b6-492f-8a1a-e93d5402b654" containerName="istio-proxy"
Oct 30 01:06:50.320670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320660    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="71d6f8ed-df9e-4e61-951d-c4490161d7ae" containerName="istio-proxy"
Oct 30 01:06:50.320743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320734    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy"
Oct 30 01:06:50.320824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320815    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f83157ed-e10d-4f90-a3bc-bec67e35b3b8" containerName="istio-proxy"
Oct 30 01:06:50.320933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.320922    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy"
Oct 30 01:06:50.330943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.330919    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.360912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.360360    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-ad449e-dockercfg-vbgrk\""
Oct 30 01:06:50.368871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.361217    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-ad449e\""
Oct 30 01:06:50.368871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.362435    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"]
Oct 30 01:06:50.559664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.559624    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.560102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.560074    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bhq6\" (UniqueName: \"kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6\") pod \"payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.611117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.611041    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"]
Oct 30 01:06:50.612913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.611701    4577 cpu_manager.go:414] "containerMap: removing container" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy"
Oct 30 01:06:50.612913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.611718    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="765e057d-2b90-442f-bec5-78841a2d9619" containerName="istio-proxy"
Oct 30 01:06:50.615587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.615563    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.616599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.616571    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"]
Oct 30 01:06:50.680591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.678997    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bhq6\" (UniqueName: \"kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6\") pod \"payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.680591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.679085    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqslc\" (UniqueName: \"kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc\") pod \"payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.680591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.679244    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.680591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.679277    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.680591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.680503    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.704827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.704802    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bhq6\" (UniqueName: \"kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6\") pod \"payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:50.781543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.781147    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.781543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.781402    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqslc\" (UniqueName: \"kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc\") pod \"payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.782556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.782529    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.807118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.807091    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqslc\" (UniqueName: \"kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc\") pod \"payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.931283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.931197    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:50.981516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:50.981473    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:51.037600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:51.037530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:51.411564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:51.411535    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:06:51.511231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:51.511198    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"]
Oct 30 01:06:51.511382 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:06:51.511201    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2b578c_c2df_41db_92a2_406a099c5638.slice/crio-45f6e8ed7b978600730cfb7e2bf278c7eebdd850df6dcc764f8831e50ce1ebf0 WatchSource:0}: Error finding container 45f6e8ed7b978600730cfb7e2bf278c7eebdd850df6dcc764f8831e50ce1ebf0: Status 404 returned error can't find the container with id 45f6e8ed7b978600730cfb7e2bf278c7eebdd850df6dcc764f8831e50ce1ebf0
Oct 30 01:06:51.526979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:51.526952    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"]
Oct 30 01:06:51.599696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:51.599662    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj" event={"ID":"3d1e47c1-33b1-4153-ae91-b44a72174b1a","Type":"ContainerStarted","Data":"e5aaa762dfabef745849111278844d0e0f9d430fee7b1289381235dcc299ac37"}
Oct 30 01:06:51.601421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:51.601353    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8" event={"ID":"4e2b578c-c2df-41db-92a2-406a099c5638","Type":"ContainerStarted","Data":"45f6e8ed7b978600730cfb7e2bf278c7eebdd850df6dcc764f8831e50ce1ebf0"}
Oct 30 01:06:52.264705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.264674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:52.411629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.411593    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:52.611467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.611382    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj" event={"ID":"3d1e47c1-33b1-4153-ae91-b44a72174b1a","Type":"ContainerStarted","Data":"3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c"}
Oct 30 01:06:52.611467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.611461    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:06:52.613335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.613302    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8" event={"ID":"4e2b578c-c2df-41db-92a2-406a099c5638","Type":"ContainerStarted","Data":"8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99"}
Oct 30 01:06:52.613470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.613367    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:06:52.670074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.670000    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj" podStartSLOduration=2.669980266 podStartE2EDuration="2.669980266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:52.66837445 +0000 UTC m=+1520.697684674" watchObservedRunningTime="2026-10-30 01:06:52.669980266 +0000 UTC m=+1520.699290500"
Oct 30 01:06:52.718428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.718374    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8" podStartSLOduration=2.7183591 podStartE2EDuration="2.7183591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:52.71757591 +0000 UTC m=+1520.746886154" watchObservedRunningTime="2026-10-30 01:06:52.7183591 +0000 UTC m=+1520.747669331"
Oct 30 01:06:52.826548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:52.826509    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:54.132337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.132289    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:54.311508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.309705    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:54.623903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.617916    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"]
Oct 30 01:06:54.649883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.641148    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.688868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.685795    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a8ab86\""
Oct 30 01:06:54.738351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.728094    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a8ab86-dockercfg-9glm5\""
Oct 30 01:06:54.749763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.749443    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.749763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.749525    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tpnz\" (UniqueName: \"kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz\") pod \"payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.823874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.818270    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"]
Oct 30 01:06:54.834813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.834775    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"]
Oct 30 01:06:54.888585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.888474    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.903001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.872930    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.903001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.897336    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpnz\" (UniqueName: \"kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz\") pod \"payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.913738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.913374    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"]
Oct 30 01:06:54.913738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.913491    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:54.923564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.923530    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tpnz\" (UniqueName: \"kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz\") pod \"payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:54.999213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.999169    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:54.999405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:54.999369    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhknl\" (UniqueName: \"kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl\") pod \"payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:55.100690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.100658    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:55.100910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.100784    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhknl\" (UniqueName: \"kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl\") pod \"payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:55.101475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.101449    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:55.118733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.118704    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:55.129070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.129027    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhknl\" (UniqueName: \"kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl\") pod \"payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:55.198947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.198911    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"]
Oct 30 01:06:55.205299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.205265    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.226023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.222647    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-838fd5-serving-cert\""
Oct 30 01:06:55.229026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.228564    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"]
Oct 30 01:06:55.229026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.228825    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:55.290668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.280767    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:55.306779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.303817    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.306779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.303919    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.306779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.304053    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.306779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.304090    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mtbn\" (UniqueName: \"kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.413920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.413456    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:55.426875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.423647    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.426875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.423736    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.426875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.423828    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.426875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.423871    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mtbn\" (UniqueName: \"kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.436867 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:55.430144    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-serving-cert: secret "maas-api-e2e-ipp-b-838fd5-serving-cert" not found
Oct 30 01:06:55.436867 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:55.430226    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls podName:e6b5ffd5-771a-46b6-8806-4fead8f98d6e nodeName:}" failed. No retries permitted until 2026-10-30 01:06:55.9302029 +0000 UTC m=+1523.959513123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls") pod "maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" (UID: "e6b5ffd5-771a-46b6-8806-4fead8f98d6e") : secret "maas-api-e2e-ipp-b-838fd5-serving-cert" not found
Oct 30 01:06:55.436867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.430997    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.531064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.531030    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.544313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.544215    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mtbn\" (UniqueName: \"kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.547515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.547484    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"]
Oct 30 01:06:55.604724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.604699    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"]
Oct 30 01:06:55.612269 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:06:55.612235    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c25040_db18_4255_9d25_7fe1a5aa0b5b.slice/crio-74ba3f65baf587ec1d7b1246f5cad2daa08bb0f9ce27a37e62e860eee50cdfc9 WatchSource:0}: Error finding container 74ba3f65baf587ec1d7b1246f5cad2daa08bb0f9ce27a37e62e860eee50cdfc9: Status 404 returned error can't find the container with id 74ba3f65baf587ec1d7b1246f5cad2daa08bb0f9ce27a37e62e860eee50cdfc9
Oct 30 01:06:55.647626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.647554    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9" event={"ID":"6c4c8502-4e79-46f5-9f41-84f65c8c3350","Type":"ContainerStarted","Data":"1b4c0fa725b586cdb470748997be5165ec405a710226b85ca30e59190c74d094"}
Oct 30 01:06:55.649091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.649068    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" event={"ID":"d7c25040-db18-4255-9d25-7fe1a5aa0b5b","Type":"ContainerStarted","Data":"74ba3f65baf587ec1d7b1246f5cad2daa08bb0f9ce27a37e62e860eee50cdfc9"}
Oct 30 01:06:55.663642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.663554    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:55.944329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.944245    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:55.951865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:55.947431    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:56.140382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.140351    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:56.574153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.573737    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"]
Oct 30 01:06:56.661189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.661150    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9" event={"ID":"6c4c8502-4e79-46f5-9f41-84f65c8c3350","Type":"ContainerStarted","Data":"ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c"}
Oct 30 01:06:56.662246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.662225    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:06:56.664262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.664152    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" event={"ID":"d7c25040-db18-4255-9d25-7fe1a5aa0b5b","Type":"ContainerStarted","Data":"ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"}
Oct 30 01:06:56.664262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.664231    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:06:56.665693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.665668    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" event={"ID":"e6b5ffd5-771a-46b6-8806-4fead8f98d6e","Type":"ContainerStarted","Data":"2ccde1166a485ed6e7ce19eda7f7984400b5a53b650003d70f1507d202ba4d82"}
Oct 30 01:06:56.708330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.708273    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9" podStartSLOduration=2.708256661 podStartE2EDuration="2.708256661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:56.707580031 +0000 UTC m=+1524.736890282" watchObservedRunningTime="2026-10-30 01:06:56.708256661 +0000 UTC m=+1524.737566895"
Oct 30 01:06:56.755959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:56.755910    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" podStartSLOduration=2.755896709 podStartE2EDuration="2.755896709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:56.752761316 +0000 UTC m=+1524.782071570" watchObservedRunningTime="2026-10-30 01:06:56.755896709 +0000 UTC m=+1524.785206939"
Oct 30 01:06:57.040912 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:06:57.040839    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:06:57.119131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:57.119097    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:57.676118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:57.676078    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" event={"ID":"e6b5ffd5-771a-46b6-8806-4fead8f98d6e","Type":"ContainerStarted","Data":"f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9"}
Oct 30 01:06:57.676612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:57.676158    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:06:57.723909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:57.723839    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" podStartSLOduration=2.327331081 podStartE2EDuration="2.723821855s" totalImagesPullingTime="396.490774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:57.722524555 +0000 UTC m=+1525.751834789" watchObservedRunningTime="2026-10-30 01:06:57.723821855 +0000 UTC m=+1525.753132086"
Oct 30 01:06:58.943141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:58.943099    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:59.116881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:59.116806    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:06:59.234005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:06:59.233979    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:00.706359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.706329    4577 generic.go:356] "Generic (PLEG): container finished" podID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerID="37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a" exitCode=0
Oct 30 01:07:00.706885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.706413    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" event={"ID":"b10f3bf4-0a71-4aad-866f-4bf48958f6b7","Type":"ContainerDied","Data":"37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"}
Oct 30 01:07:00.706885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.706878    4577 scope.go:122] "RemoveContainer" containerID="37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"
Oct 30 01:07:00.710023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.709991    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:00.710162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.710076    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:00.710162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.710128    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:00.782355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.782331    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:00.906432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:00.906396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:01.716446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:01.716417    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" event={"ID":"b10f3bf4-0a71-4aad-866f-4bf48958f6b7","Type":"ContainerStarted","Data":"e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a"}
Oct 30 01:07:02.078386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:02.078346    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:07:02.079766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:02.079736    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 01:07:02.079932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:02.079787    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 01:07:02.692314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:02.692278    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:03.078733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:03.078700    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 01:07:03.079248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:03.078758    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 01:07:03.622948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:03.622916    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:07:03.623128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:03.623003    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:07:03.691031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:03.691003    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:07:04.078463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:04.078426    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 01:07:04.078696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:04.078507    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 01:07:04.468926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:04.455377    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:05.078621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:05.078583    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 01:07:05.078818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:05.078672    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 01:07:06.078139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:06.078046    4577 patch_prober.go:29] interesting pod/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 01:07:06.078139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:06.078107    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 01:07:06.680782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:06.680739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:06.794865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:06.794729    4577 generic.go:356] "Generic (PLEG): container finished" podID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerID="ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79" exitCode=1
Oct 30 01:07:06.794865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:06.794801    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" event={"ID":"d7c25040-db18-4255-9d25-7fe1a5aa0b5b","Type":"ContainerDied","Data":"ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"}
Oct 30 01:07:06.800868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:06.795373    4577 scope.go:122] "RemoveContainer" containerID="ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"
Oct 30 01:07:07.097069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:07.097002    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:07:07.097602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:07.097578    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:07:07.098807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:07.098776    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:07:07.805138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:07.805107    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" event={"ID":"d7c25040-db18-4255-9d25-7fe1a5aa0b5b","Type":"ContainerStarted","Data":"137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471"}
Oct 30 01:07:07.805355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:07.805338    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:07:08.419618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:08.419588    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:08.688028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:08.687929    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:07:10.747877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:10.747809    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:13.023342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:13.023306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:14.744098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:14.744065    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:16.959435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:16.959396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:18.686903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:18.686867    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:18.820086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:18.820046    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:07:20.124013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:20.123971    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:07:20.124405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:20.124040    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:07:20.584023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:20.583988    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:20.696177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:20.695363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:20.978897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:20.978868    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:21.092291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:21.092257    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:21.297073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:21.296369    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"]
Oct 30 01:07:21.395543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:21.395467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:21.517094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:21.516978    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:21.638998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:21.638915    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:22.005770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.005735    4577 generic.go:356] "Generic (PLEG): container finished" podID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerID="6a2a86556eba28fa9e9b81775485d9b45ec369a01869bca446973977784aa3ff" exitCode=0
Oct 30 01:07:22.006243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.006217    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" event={"ID":"47dcdb1c-2440-4ffc-aecd-8f7dae131d12","Type":"ContainerDied","Data":"6a2a86556eba28fa9e9b81775485d9b45ec369a01869bca446973977784aa3ff"}
Oct 30 01:07:22.007061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.007012    4577 scope.go:122] "RemoveContainer" containerID="6a2a86556eba28fa9e9b81775485d9b45ec369a01869bca446973977784aa3ff"
Oct 30 01:07:22.010690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.010632    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:22.011040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.011015    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:22.011440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.011415    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:22.497706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:22.497676    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:23.039005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:23.038965    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" event={"ID":"47dcdb1c-2440-4ffc-aecd-8f7dae131d12","Type":"ContainerStarted","Data":"677b0e21da126cf5914dd3959f8658590499d9ecf7e1fb3143b609a9b495f59d"}
Oct 30 01:07:23.752872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:23.749215    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:07:23.752872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:23.749343    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:07:23.752872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:23.749393    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:07:23.975911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:23.971943    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:24.307860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:24.307817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:24.763190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:24.763157    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:07:24.763767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:24.763738    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:07:25.761872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:25.749155    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:07:25.761872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:25.749225    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:07:25.993521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:25.993481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:26.297529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.297486    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc" podUID="e5c9ea7a-bb62-4353-ae53-0b0c3d887895" containerName="payload-pre-processing" containerID="cri-o://9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07" gracePeriod=25
Oct 30 01:07:26.638885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.638860    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:07:26.734756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.734719    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume\") pod \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") "
Oct 30 01:07:26.734944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.734811    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29\" (UniqueName: \"kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29\") pod \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\" (UID: \"e5c9ea7a-bb62-4353-ae53-0b0c3d887895\") "
Oct 30 01:07:26.735215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.735185    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume" pod "e5c9ea7a-bb62-4353-ae53-0b0c3d887895" (UID: "e5c9ea7a-bb62-4353-ae53-0b0c3d887895"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:07:26.735575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.735554    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:26.737301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.737270    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29" pod "e5c9ea7a-bb62-4353-ae53-0b0c3d887895" (UID: "e5c9ea7a-bb62-4353-ae53-0b0c3d887895"). InnerVolumeSpecName "kube-api-access-nwc29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:26.755204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.755179    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:07:26.755405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.755381    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:07:26.837448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:26.837412    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwc29\" (UniqueName: \"kubernetes.io/projected/e5c9ea7a-bb62-4353-ae53-0b0c3d887895-kube-api-access-nwc29\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:27.023625 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:07:27.023586    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:07:27.133835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.133799    4577 generic.go:356] "Generic (PLEG): container finished" podID="e5c9ea7a-bb62-4353-ae53-0b0c3d887895" containerID="9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07" exitCode=0
Oct 30 01:07:27.134038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.133872    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"
Oct 30 01:07:27.134038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.133882    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc" event={"ID":"e5c9ea7a-bb62-4353-ae53-0b0c3d887895","Type":"ContainerDied","Data":"9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07"}
Oct 30 01:07:27.134038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.133952    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc" event={"ID":"e5c9ea7a-bb62-4353-ae53-0b0c3d887895","Type":"ContainerDied","Data":"ada0514401f8513454e9fc65b107384b8546522039c3b2810e56309832c2bb58"}
Oct 30 01:07:27.134038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.133981    4577 scope.go:122] "RemoveContainer" containerID="9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07"
Oct 30 01:07:27.144833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.144816    4577 scope.go:122] "RemoveContainer" containerID="9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07"
Oct 30 01:07:27.163012 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:07:27.161981    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07\": container with ID starting with 9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07 not found: ID does not exist" containerID="9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07"
Oct 30 01:07:27.163012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.162028    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07"} err="failed to get container status \"9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07\": rpc error: code = NotFound desc = could not find container \"9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07\": container with ID starting with 9ff8a3a1ca7162ef5875ad696eb21d84a18012494212b421b6b3b5bd3a5c0d07 not found: ID does not exist"
Oct 30 01:07:27.205861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.205815    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"]
Oct 30 01:07:27.215072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.215037    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-838fd5-578c468f7f-m29bc"]
Oct 30 01:07:27.744653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.744617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:27.752220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.752187    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:07:27.752431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:27.752406    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:07:28.111984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:28.111910    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5c9ea7a-bb62-4353-ae53-0b0c3d887895" path="/var/lib/kubelet/pods/e5c9ea7a-bb62-4353-ae53-0b0c3d887895/volumes"
Oct 30 01:07:28.752600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:28.752572    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:07:28.752887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:28.752866    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:07:28.753690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:28.753668    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:07:30.006696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:30.006658    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:31.796780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:31.796734    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:34.122455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.122406    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:34.371128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.371096    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"]
Oct 30 01:07:34.371935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.371914    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e5c9ea7a-bb62-4353-ae53-0b0c3d887895" containerName="payload-pre-processing"
Oct 30 01:07:34.371935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.371936    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5c9ea7a-bb62-4353-ae53-0b0c3d887895" containerName="payload-pre-processing"
Oct 30 01:07:34.372163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.372148    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5c9ea7a-bb62-4353-ae53-0b0c3d887895" containerName="payload-pre-processing"
Oct 30 01:07:34.403926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.403838    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"]
Oct 30 01:07:34.404250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.404179    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.428234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.420923    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-429c67-openshift-default-dockercfg-bh6ns\""
Oct 30 01:07:34.428234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.424902    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:34.544057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.544000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:34.590628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.590592    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.591534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.591509    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlhs6\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.591777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.591759    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.592135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.592116    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.592316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.592300    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.592506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.592491    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.592694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.592678    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.592881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.592863    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.593083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.593067    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.696041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.695945    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.696377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.696354    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlhs6\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.696524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.696507    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.696638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.696622    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.696743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.696730    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.696954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.696933    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.697111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.697095    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.697257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.697241    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.697442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.697424    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.698370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.698349    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.698778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.698758    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.705067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.705047    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.707304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.706538    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.715722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.707728    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.718799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.718773    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.739943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.739910    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.741567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.741509    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlhs6\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:34.747036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:34.746980    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token\") pod \"e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") " pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:35.022962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:35.022934    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:35.354052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:35.354022    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"]
Oct 30 01:07:35.354792 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:07:35.354769    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19927d1_bfcb_4d1f_8a1d_79e9beed85f0.slice/crio-5f63c8b3e8bc9a089fd8b0b8b308780fda263c5b0a0401c54664ecd47aae7f3a WatchSource:0}: Error finding container 5f63c8b3e8bc9a089fd8b0b8b308780fda263c5b0a0401c54664ecd47aae7f3a: Status 404 returned error can't find the container with id 5f63c8b3e8bc9a089fd8b0b8b308780fda263c5b0a0401c54664ecd47aae7f3a
Oct 30 01:07:35.357500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:35.357468    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:35.357607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:35.357532    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:35.357607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:35.357563    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:07:36.232152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:36.232117    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" event={"ID":"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0","Type":"ContainerStarted","Data":"efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"}
Oct 30 01:07:36.232342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:36.232177    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" event={"ID":"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0","Type":"ContainerStarted","Data":"5f63c8b3e8bc9a089fd8b0b8b308780fda263c5b0a0401c54664ecd47aae7f3a"}
Oct 30 01:07:36.288524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:36.288470    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podStartSLOduration=2.288456123 podStartE2EDuration="2.288456123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:07:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:07:36.284466718 +0000 UTC m=+1564.313776951" watchObservedRunningTime="2026-10-30 01:07:36.288456123 +0000 UTC m=+1564.317766354"
Oct 30 01:07:36.504250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:36.504157    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:37.023644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:37.023608    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:37.029830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:37.029809    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:37.240407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:37.240379    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:37.241387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:37.241367    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:07:38.387465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:38.387432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:38.684885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:38.684790    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:38.817714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:38.817673    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:40.288711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:40.288679    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:40.414860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:40.414766    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:40.537803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:40.537769    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:40.773775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:40.773747    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:41.032445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:41.032364    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:41.243275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:41.243239    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:42.437634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:42.437587    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:42.625890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:42.614146    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:43.047955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.047919    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"]
Oct 30 01:07:43.075530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.075495    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"]
Oct 30 01:07:43.075704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.075652    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.098213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.098179    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-838fd5\""
Oct 30 01:07:43.098378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.098365    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-838fd5-dockercfg-f4zxz\""
Oct 30 01:07:43.171034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.170988    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.171297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.171269    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s74gv\" (UniqueName: \"kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv\") pod \"payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.272582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.272535    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s74gv\" (UniqueName: \"kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv\") pod \"payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.272774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.272755    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.273652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.273628    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.298184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.298130    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s74gv\" (UniqueName: \"kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv\") pod \"payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.410227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.410186    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:43.722778 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:07:43.722748    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e1f7c9_a823_41cc_9417_a81e702f3e3f.slice/crio-bafd8978d17ccbbe65c6465ebe3c8ac7ccca1caaab380a94ff0aac55a284ac31 WatchSource:0}: Error finding container bafd8978d17ccbbe65c6465ebe3c8ac7ccca1caaab380a94ff0aac55a284ac31: Status 404 returned error can't find the container with id bafd8978d17ccbbe65c6465ebe3c8ac7ccca1caaab380a94ff0aac55a284ac31
Oct 30 01:07:43.723151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.722990    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"]
Oct 30 01:07:43.749369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:43.749343    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:44.336593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:44.336556    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6" event={"ID":"95e1f7c9-a823-41cc-9417-a81e702f3e3f","Type":"ContainerStarted","Data":"d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d"}
Oct 30 01:07:44.336804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:44.336629    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6" event={"ID":"95e1f7c9-a823-41cc-9417-a81e702f3e3f","Type":"ContainerStarted","Data":"bafd8978d17ccbbe65c6465ebe3c8ac7ccca1caaab380a94ff0aac55a284ac31"}
Oct 30 01:07:44.336804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:44.336766    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:44.383833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:44.383774    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6" podStartSLOduration=2.383761368 podStartE2EDuration="2.383761368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:07:44.380799525 +0000 UTC m=+1572.410109758" watchObservedRunningTime="2026-10-30 01:07:44.383761368 +0000 UTC m=+1572.413071598"
Oct 30 01:07:44.728770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:44.728563    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:44.846213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:44.846172    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:45.732898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:45.732836    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:46.338462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:46.338426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:46.454318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:46.454280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:47.706284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:47.706249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:48.729081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:48.729049    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:49.601653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:49.601614    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:49.720109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:49.720074    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:50.021716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:50.021687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:50.142491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:50.142457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:50.394294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:50.394179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:51.505298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:51.505262    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:53.090999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.090966    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"]
Oct 30 01:07:53.091442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.091257    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" podUID="ccd2c83f-436f-465f-95c9-bf74a557f12a" containerName="maas-api" containerID="cri-o://f47e28fc8f82a8103c9993fdfd7eabfbe3f0ae1343202f8c658332e4c9235bab" gracePeriod=30
Oct 30 01:07:53.166472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.161783    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:53.253817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.253786    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"]
Oct 30 01:07:53.254039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.254018    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h" podUID="12af8977-9c14-4f55-806f-8753b42536e2" containerName="payload-processing" containerID="cri-o://1efccff75ff779cc0b1c8a6291b800c3f82f159528042c67cb184882349afa94" gracePeriod=30
Oct 30 01:07:53.285336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.284995    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"]
Oct 30 01:07:53.285715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.285685    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td" podUID="00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" containerName="payload-pre-processing" containerID="cri-o://2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3" gracePeriod=30
Oct 30 01:07:53.335937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.335891    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:53.413911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.411035    4577 generic.go:356] "Generic (PLEG): container finished" podID="12af8977-9c14-4f55-806f-8753b42536e2" containerID="1efccff75ff779cc0b1c8a6291b800c3f82f159528042c67cb184882349afa94" exitCode=0
Oct 30 01:07:53.413911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.411342    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h" event={"ID":"12af8977-9c14-4f55-806f-8753b42536e2","Type":"ContainerDied","Data":"1efccff75ff779cc0b1c8a6291b800c3f82f159528042c67cb184882349afa94"}
Oct 30 01:07:53.413911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.413901    4577 generic.go:356] "Generic (PLEG): container finished" podID="ccd2c83f-436f-465f-95c9-bf74a557f12a" containerID="f47e28fc8f82a8103c9993fdfd7eabfbe3f0ae1343202f8c658332e4c9235bab" exitCode=0
Oct 30 01:07:53.416108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.413962    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" event={"ID":"ccd2c83f-436f-465f-95c9-bf74a557f12a","Type":"ContainerDied","Data":"f47e28fc8f82a8103c9993fdfd7eabfbe3f0ae1343202f8c658332e4c9235bab"}
Oct 30 01:07:53.451871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.450367    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:53.466304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.466271    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:07:53.570998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.570924    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:07:53.630921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.630256    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk\" (UniqueName: \"kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk\") pod \"ccd2c83f-436f-465f-95c9-bf74a557f12a\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") "
Oct 30 01:07:53.630921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.630442    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs\") pod \"ccd2c83f-436f-465f-95c9-bf74a557f12a\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") "
Oct 30 01:07:53.630921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.630575    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca\") pod \"ccd2c83f-436f-465f-95c9-bf74a557f12a\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") "
Oct 30 01:07:53.630921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.630603    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\") pod \"ccd2c83f-436f-465f-95c9-bf74a557f12a\" (UID: \"ccd2c83f-436f-465f-95c9-bf74a557f12a\") "
Oct 30 01:07:53.638405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.638353    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca" pod "ccd2c83f-436f-465f-95c9-bf74a557f12a" (UID: "ccd2c83f-436f-465f-95c9-bf74a557f12a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:07:53.683870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.657796    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd2c83f-436f-465f-95c9-bf74a557f12a-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:53.713891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.712308    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:53.790914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.768102    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w\" (UniqueName: \"kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w\") pod \"12af8977-9c14-4f55-806f-8753b42536e2\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") "
Oct 30 01:07:53.790914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.768302    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume\") pod \"12af8977-9c14-4f55-806f-8753b42536e2\" (UID: \"12af8977-9c14-4f55-806f-8753b42536e2\") "
Oct 30 01:07:53.790914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.785328    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs" pod "ccd2c83f-436f-465f-95c9-bf74a557f12a" (UID: "ccd2c83f-436f-465f-95c9-bf74a557f12a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:07:53.792462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.791448    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume" pod "12af8977-9c14-4f55-806f-8753b42536e2" (UID: "12af8977-9c14-4f55-806f-8753b42536e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:07:53.840867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.840755    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:53.858107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.849990    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls" pod "ccd2c83f-436f-465f-95c9-bf74a557f12a" (UID: "ccd2c83f-436f-465f-95c9-bf74a557f12a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:07:53.948607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.948528    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:53.948607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.948563    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccd2c83f-436f-465f-95c9-bf74a557f12a-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:53.948607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.948578    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12af8977-9c14-4f55-806f-8753b42536e2-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:53.949133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.949107    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk" pod "ccd2c83f-436f-465f-95c9-bf74a557f12a" (UID: "ccd2c83f-436f-465f-95c9-bf74a557f12a"). InnerVolumeSpecName "kube-api-access-hwtmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:53.959936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.959082    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w" pod "12af8977-9c14-4f55-806f-8753b42536e2" (UID: "12af8977-9c14-4f55-806f-8753b42536e2"). InnerVolumeSpecName "kube-api-access-9n22w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:53.981658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:53.981624    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:07:54.054913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.054330    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5\" (UniqueName: \"kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5\") pod \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") "
Oct 30 01:07:54.054913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.054630    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume\") pod \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\" (UID: \"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5\") "
Oct 30 01:07:54.066927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.055329    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwtmk\" (UniqueName: \"kubernetes.io/projected/ccd2c83f-436f-465f-95c9-bf74a557f12a-kube-api-access-hwtmk\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:54.066927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.055355    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n22w\" (UniqueName: \"kubernetes.io/projected/12af8977-9c14-4f55-806f-8753b42536e2-kube-api-access-9n22w\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:54.075577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.075535    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume" pod "00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" (UID: "00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:07:54.076537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.076509    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5" pod "00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" (UID: "00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5"). InnerVolumeSpecName "kube-api-access-j8qq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:54.156194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.156166    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:54.156194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.156195    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8qq5\" (UniqueName: \"kubernetes.io/projected/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5-kube-api-access-j8qq5\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:07:54.333641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.333603    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:54.422607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.422573    4577 generic.go:356] "Generic (PLEG): container finished" podID="00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" containerID="2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3" exitCode=0
Oct 30 01:07:54.422799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.422667    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"
Oct 30 01:07:54.422799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.422678    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td" event={"ID":"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5","Type":"ContainerDied","Data":"2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3"}
Oct 30 01:07:54.422799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.422740    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td" event={"ID":"00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5","Type":"ContainerDied","Data":"b7e774f759a05f476ff5ffb88886033cd2ca901abeb58f228c87c61b8c005ef6"}
Oct 30 01:07:54.422799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.422764    4577 scope.go:122] "RemoveContainer" containerID="2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3"
Oct 30 01:07:54.425206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.425184    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"
Oct 30 01:07:54.425368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.425187    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk" event={"ID":"ccd2c83f-436f-465f-95c9-bf74a557f12a","Type":"ContainerDied","Data":"da996b60dbbfbbf3fee3e89332c888128ddf158ceaa90693705981e2b87a393a"}
Oct 30 01:07:54.426910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.426884    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h" event={"ID":"12af8977-9c14-4f55-806f-8753b42536e2","Type":"ContainerDied","Data":"e9698cf64960f05cd626d5e8b688ef1c4f6a127d7ffe72e89c8ae2c90a123a2a"}
Oct 30 01:07:54.427025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.426900    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"
Oct 30 01:07:54.435041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.435020    4577 scope.go:122] "RemoveContainer" containerID="2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3"
Oct 30 01:07:54.435314 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:07:54.435295    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3\": container with ID starting with 2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3 not found: ID does not exist" containerID="2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3"
Oct 30 01:07:54.435375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.435322    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3"} err="failed to get container status \"2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3\": rpc error: code = NotFound desc = could not find container \"2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3\": container with ID starting with 2e28d0f6dbc5980ec917df0a2abc03ef9f776a09420350743c0fc88e55c0e7b3 not found: ID does not exist"
Oct 30 01:07:54.435375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.435339    4577 scope.go:122] "RemoveContainer" containerID="f47e28fc8f82a8103c9993fdfd7eabfbe3f0ae1343202f8c658332e4c9235bab"
Oct 30 01:07:54.445168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.445146    4577 scope.go:122] "RemoveContainer" containerID="1efccff75ff779cc0b1c8a6291b800c3f82f159528042c67cb184882349afa94"
Oct 30 01:07:54.484501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.484453    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"]
Oct 30 01:07:54.489072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.489048    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d4622a-79dcc6564d-mz82h"]
Oct 30 01:07:54.538976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.538950    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"]
Oct 30 01:07:54.542622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.542591    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d4622a-6db5d464cf-dv9td"]
Oct 30 01:07:54.578743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.578709    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"]
Oct 30 01:07:54.582926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:54.582905    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d4622a-7698ddf7df-br9xk"]
Oct 30 01:07:55.345447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:55.345417    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:07:55.859790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:55.859759    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:07:56.112631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:56.112510    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" path="/var/lib/kubelet/pods/00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5/volumes"
Oct 30 01:07:56.113119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:56.113103    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12af8977-9c14-4f55-806f-8753b42536e2" path="/var/lib/kubelet/pods/12af8977-9c14-4f55-806f-8753b42536e2/volumes"
Oct 30 01:07:56.113610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:56.113593    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ccd2c83f-436f-465f-95c9-bf74a557f12a" path="/var/lib/kubelet/pods/ccd2c83f-436f-465f-95c9-bf74a557f12a/volumes"
Oct 30 01:07:57.013772 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:07:57.013736    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:07:58.334436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:07:58.334403    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:00.844579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:00.844537    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:00.996872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:00.996049    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:01.110588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:01.110189    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:02.602568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.602537    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"]
Oct 30 01:08:02.603234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603215    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ccd2c83f-436f-465f-95c9-bf74a557f12a" containerName="maas-api"
Oct 30 01:08:02.603234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603234    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ccd2c83f-436f-465f-95c9-bf74a557f12a" containerName="maas-api"
Oct 30 01:08:02.603367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603254    4577 cpu_manager.go:414] "containerMap: removing container" podUID="12af8977-9c14-4f55-806f-8753b42536e2" containerName="payload-processing"
Oct 30 01:08:02.603367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603264    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12af8977-9c14-4f55-806f-8753b42536e2" containerName="payload-processing"
Oct 30 01:08:02.603367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603295    4577 cpu_manager.go:414] "containerMap: removing container" podUID="00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" containerName="payload-pre-processing"
Oct 30 01:08:02.603367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603301    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" containerName="payload-pre-processing"
Oct 30 01:08:02.603609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603409    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="00cc2df4-8523-4fa6-ab4c-f7accbb3a4b5" containerName="payload-pre-processing"
Oct 30 01:08:02.603609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603423    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ccd2c83f-436f-465f-95c9-bf74a557f12a" containerName="maas-api"
Oct 30 01:08:02.603609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.603431    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="12af8977-9c14-4f55-806f-8753b42536e2" containerName="payload-processing"
Oct 30 01:08:02.622385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.622333    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.636518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.636486    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"]
Oct 30 01:08:02.642951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.642920    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-429c67-serving-cert\""
Oct 30 01:08:02.718693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.718666    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:02.786392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.786351    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.786580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.786451    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.786774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.786748    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.786835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.786814    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhhjn\" (UniqueName: \"kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.859964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.856286    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:02.918073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.918028    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.918369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.918113    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.918533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.918463    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.918915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.918837    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.923582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.918519    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhhjn\" (UniqueName: \"kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.957024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.953804    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.958867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.958373    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:02.963436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:02.963400    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhhjn\" (UniqueName: \"kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn\") pod \"maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:03.252067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:03.252035    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:03.573995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:03.573967    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"]
Oct 30 01:08:03.576542 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:03.576514    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37ccf5c_5855_4b20_9b6c_f6ff3d5e98b1.slice/crio-7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c WatchSource:0}: Error finding container 7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c: Status 404 returned error can't find the container with id 7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c
Oct 30 01:08:04.529500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:04.529464    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl" event={"ID":"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1","Type":"ContainerStarted","Data":"799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7"}
Oct 30 01:08:04.530014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:04.529529    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:04.530014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:04.529551    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl" event={"ID":"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1","Type":"ContainerStarted","Data":"7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c"}
Oct 30 01:08:04.578807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:04.578758    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl" podStartSLOduration=2.204738754 podStartE2EDuration="2.57874437s" totalImagesPullingTime="374.005616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:04.574875345 +0000 UTC m=+1592.604185573" watchObservedRunningTime="2026-10-30 01:08:04.57874437 +0000 UTC m=+1592.608054600"
Oct 30 01:08:04.643014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:04.642975    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:07.065551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:07.065509    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:09.353032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.353003    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"]
Oct 30 01:08:09.357661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.357638    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.379342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.379312    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"]
Oct 30 01:08:09.390349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.389982    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-b43bff-serving-cert\""
Oct 30 01:08:09.443384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.443346    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.443595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.443571    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.443692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.443675    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.443770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.443754    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqh6s\" (UniqueName: \"kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.457475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.457439    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:09.563417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.563373    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.573869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.563709    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.574217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.574187    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqh6s\" (UniqueName: \"kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.574535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.574516    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.587660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.579718    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.587922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.570643    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.597870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.594556    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:09.651718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.651642    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqh6s\" (UniqueName: \"kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.668633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.667797    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.673934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.673872    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:09.741904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.741776    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:09.984483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:09.984454    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"]
Oct 30 01:08:09.985545 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:09.985516    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311b2aa4_f32a_4482_9c7d_207f52e599f2.slice/crio-6eb010b47ee222a7f6ce4a1029ef9fed15a87c3405b863afd245f2f8a6c3ec87 WatchSource:0}: Error finding container 6eb010b47ee222a7f6ce4a1029ef9fed15a87c3405b863afd245f2f8a6c3ec87: Status 404 returned error can't find the container with id 6eb010b47ee222a7f6ce4a1029ef9fed15a87c3405b863afd245f2f8a6c3ec87
Oct 30 01:08:10.544280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:10.544250    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:08:10.794084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:10.793375    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc" event={"ID":"311b2aa4-f32a-4482-9c7d-207f52e599f2","Type":"ContainerStarted","Data":"f2cf9a994ad4d03c14686e13a4779e9db532f28cc6974274886b1652f740940c"}
Oct 30 01:08:10.794084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:10.793439    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc" event={"ID":"311b2aa4-f32a-4482-9c7d-207f52e599f2","Type":"ContainerStarted","Data":"6eb010b47ee222a7f6ce4a1029ef9fed15a87c3405b863afd245f2f8a6c3ec87"}
Oct 30 01:08:10.795013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:10.794520    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:10.849895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:10.847092    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc" podStartSLOduration=1.43037557 podStartE2EDuration="1.847072594s" totalImagesPullingTime="416.697024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:10.841643762 +0000 UTC m=+1598.870953996" watchObservedRunningTime="2026-10-30 01:08:10.847072594 +0000 UTC m=+1598.876382827"
Oct 30 01:08:11.307152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:11.307119    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"]
Oct 30 01:08:11.307463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:11.307433    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy" containerID="cri-o://3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec" gracePeriod=30
Oct 30 01:08:11.368511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:11.368469    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:11.496882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:11.495891    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:13.299355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:13.299320    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:14.679771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:14.679735    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:14.895927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:14.895889    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:15.926142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:15.926105    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:16.597039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.597004    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:08:16.685508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685479    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.685715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685599    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.685791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685745    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.685791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685776    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.685924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685821    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.685924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685896    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.685924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.685912    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.686238    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.686247    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.686313    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.686470    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token\") pod \"421db701-5ef1-4a24-8fc6-342e88a351bf\" (UID: \"421db701-5ef1-4a24-8fc6-342e88a351bf\") "
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.686570    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.686808    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.687248    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.687280    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.687301    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.687315    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/421db701-5ef1-4a24-8fc6-342e88a351bf-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.687348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.687328    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.689036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.689012    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "kube-api-access-t6skp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:08:16.689160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.689058    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:08:16.693615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.690916    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:08:16.697231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.697207    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy" pod "421db701-5ef1-4a24-8fc6-342e88a351bf" (UID: "421db701-5ef1-4a24-8fc6-342e88a351bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:16.788687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.788646    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.788687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.788679    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.788687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.788690    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.788978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.788706    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6skp\" (UniqueName: \"kubernetes.io/projected/421db701-5ef1-4a24-8fc6-342e88a351bf-kube-api-access-t6skp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.788978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.788718    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/421db701-5ef1-4a24-8fc6-342e88a351bf-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:16.976575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.976480    4577 generic.go:356] "Generic (PLEG): container finished" podID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerID="3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec" exitCode=0
Oct 30 01:08:16.976575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.976531    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"
Oct 30 01:08:16.977111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.976574    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" event={"ID":"421db701-5ef1-4a24-8fc6-342e88a351bf","Type":"ContainerDied","Data":"3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec"}
Oct 30 01:08:16.977111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.976627    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9" event={"ID":"421db701-5ef1-4a24-8fc6-342e88a351bf","Type":"ContainerDied","Data":"8faaefd7f0a3229c6e0c25835c3978bf9eafda7f8467db0a9ac69dc1c43e111e"}
Oct 30 01:08:16.977111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.976649    4577 scope.go:122] "RemoveContainer" containerID="3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec"
Oct 30 01:08:16.988726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.988710    4577 scope.go:122] "RemoveContainer" containerID="e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"
Oct 30 01:08:16.999117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.999097    4577 scope.go:122] "RemoveContainer" containerID="3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec"
Oct 30 01:08:16.999447 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:16.999425    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec\": container with ID starting with 3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec not found: ID does not exist" containerID="3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec"
Oct 30 01:08:16.999537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.999457    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec"} err="failed to get container status \"3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec\": rpc error: code = NotFound desc = could not find container \"3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec\": container with ID starting with 3119ba7754d85b5da261ab508cb953d5cc917956e651920a66771cb508298eec not found: ID does not exist"
Oct 30 01:08:16.999537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.999480    4577 scope.go:122] "RemoveContainer" containerID="e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"
Oct 30 01:08:16.999786 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:16.999768    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2\": container with ID starting with e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2 not found: ID does not exist" containerID="e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"
Oct 30 01:08:16.999831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:16.999798    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2"} err="failed to get container status \"e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2\": rpc error: code = NotFound desc = could not find container \"e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2\": container with ID starting with e2ea513667f8c741747a7466be1149a06b0131e581aa4be11e13454362cebdb2 not found: ID does not exist"
Oct 30 01:08:17.047100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:17.047072    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"]
Oct 30 01:08:17.050605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:17.050580    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-d4622a-openshift-default-74bc8d8cb5-qlfs9"]
Oct 30 01:08:17.776363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:17.776323    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:17.816428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:17.815578    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:08:18.112207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:18.112134    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" path="/var/lib/kubelet/pods/421db701-5ef1-4a24-8fc6-342e88a351bf/volumes"
Oct 30 01:08:19.535360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:19.535324    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:21.212665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:21.212625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:21.372244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:21.372213    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:22.960902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:22.960831    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"]
Oct 30 01:08:22.961393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:22.961081    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr" podUID="e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" containerName="maas-api" containerID="cri-o://20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2" gracePeriod=30
Oct 30 01:08:23.115153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.115116    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"]
Oct 30 01:08:23.115320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.115301    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m" podUID="274b1218-9bc3-4101-b090-60f03462c303" containerName="payload-processing" containerID="cri-o://68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727" gracePeriod=30
Oct 30 01:08:23.121970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.121941    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:23.248933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.248715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:23.301757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.301143    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:08:23.381610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.381575    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:23.384782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.384753    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw\" (UniqueName: \"kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw\") pod \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") "
Oct 30 01:08:23.384965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.384932    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls\") pod \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") "
Oct 30 01:08:23.385062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.384977    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca\") pod \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") "
Oct 30 01:08:23.385167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.385149    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs\") pod \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\" (UID: \"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8\") "
Oct 30 01:08:23.386839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.386728    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca" pod "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" (UID: "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:08:23.388282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.388254    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls" pod "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" (UID: "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:08:23.391618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.389767    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs" pod "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" (UID: "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:08:23.391618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.391251    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw" pod "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" (UID: "e4811ec2-ed4f-4afa-9e94-a6e33b9624b8"). InnerVolumeSpecName "kube-api-access-vswvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:08:23.416559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.416526    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:08:23.514799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.488300    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume\") pod \"274b1218-9bc3-4101-b090-60f03462c303\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") "
Oct 30 01:08:23.514799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.488507    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f\" (UniqueName: \"kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f\") pod \"274b1218-9bc3-4101-b090-60f03462c303\" (UID: \"274b1218-9bc3-4101-b090-60f03462c303\") "
Oct 30 01:08:23.518038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.516095    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume" pod "274b1218-9bc3-4101-b090-60f03462c303" (UID: "274b1218-9bc3-4101-b090-60f03462c303"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:08:23.518038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.516300    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:23.518038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.516329    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vswvw\" (UniqueName: \"kubernetes.io/projected/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-kube-api-access-vswvw\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:23.518038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.516342    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:23.518038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.516356    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:23.666023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.640782    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274b1218-9bc3-4101-b090-60f03462c303-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:23.673234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.666368    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f" pod "274b1218-9bc3-4101-b090-60f03462c303" (UID: "274b1218-9bc3-4101-b090-60f03462c303"). InnerVolumeSpecName "kube-api-access-vff4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:08:23.673234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.666662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:23.743319 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.742599    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vff4f\" (UniqueName: \"kubernetes.io/projected/274b1218-9bc3-4101-b090-60f03462c303-kube-api-access-vff4f\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:23.883602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.883459    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:23.997045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:23.997013    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:24.107514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.107476    4577 generic.go:356] "Generic (PLEG): container finished" podID="e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" containerID="20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2" exitCode=0
Oct 30 01:08:24.107884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.107863    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"
Oct 30 01:08:24.110051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.110015    4577 generic.go:356] "Generic (PLEG): container finished" podID="274b1218-9bc3-4101-b090-60f03462c303" containerID="68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727" exitCode=0
Oct 30 01:08:24.110505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.110483    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"
Oct 30 01:08:24.128071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.128031    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr" event={"ID":"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8","Type":"ContainerDied","Data":"20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2"}
Oct 30 01:08:24.128495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.128474    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr" event={"ID":"e4811ec2-ed4f-4afa-9e94-a6e33b9624b8","Type":"ContainerDied","Data":"a2f0d4421472e65d0b04210196ebfa555335b2d997c0fb85f57285bd20e90dad"}
Oct 30 01:08:24.128564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.128522    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m" event={"ID":"274b1218-9bc3-4101-b090-60f03462c303","Type":"ContainerDied","Data":"68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727"}
Oct 30 01:08:24.128564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.128559    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m" event={"ID":"274b1218-9bc3-4101-b090-60f03462c303","Type":"ContainerDied","Data":"d84b0b52c40eeb36ac7f76d4d6f7496d6242c9951686e07bc0f62132acc5e6f6"}
Oct 30 01:08:24.128699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.128600    4577 scope.go:122] "RemoveContainer" containerID="20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2"
Oct 30 01:08:24.148158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.148141    4577 scope.go:122] "RemoveContainer" containerID="20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2"
Oct 30 01:08:24.148395 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:24.148375    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2\": container with ID starting with 20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2 not found: ID does not exist" containerID="20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2"
Oct 30 01:08:24.148452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.148409    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2"} err="failed to get container status \"20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2\": rpc error: code = NotFound desc = could not find container \"20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2\": container with ID starting with 20b1185c0bed98940bfbf8134068dc4f866f17065fde56e3b91a75ddf48c45e2 not found: ID does not exist"
Oct 30 01:08:24.148452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.148431    4577 scope.go:122] "RemoveContainer" containerID="68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727"
Oct 30 01:08:24.159313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.159295    4577 scope.go:122] "RemoveContainer" containerID="68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727"
Oct 30 01:08:24.159587 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:24.159565    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727\": container with ID starting with 68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727 not found: ID does not exist" containerID="68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727"
Oct 30 01:08:24.159680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.159590    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727"} err="failed to get container status \"68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727\": rpc error: code = NotFound desc = could not find container \"68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727\": container with ID starting with 68441b7e57217884b2cd760335a89bcfc531461d1b34221e640bb16e373ba727 not found: ID does not exist"
Oct 30 01:08:24.193534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.193512    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"]
Oct 30 01:08:24.200478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.200457    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-11fa15-655b967fc6-b5l8m"]
Oct 30 01:08:24.237930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.237900    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"]
Oct 30 01:08:24.244185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.244155    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fa15-58cf78697-wggsr"]
Oct 30 01:08:24.331583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:24.331543    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:26.112473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:26.112445    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="274b1218-9bc3-4101-b090-60f03462c303" path="/var/lib/kubelet/pods/274b1218-9bc3-4101-b090-60f03462c303/volumes"
Oct 30 01:08:26.113018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:26.113002    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" path="/var/lib/kubelet/pods/e4811ec2-ed4f-4afa-9e94-a6e33b9624b8/volumes"
Oct 30 01:08:26.462833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:26.462718    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:27.014508 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:27.014471    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:08:28.236047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:28.235989    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:30.071285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:30.071249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:30.615088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:30.615054    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:30.751911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:30.745226    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:32.082112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.082071    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:32.694145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.694094    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"]
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.697698    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" containerName="maas-api"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.697729    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" containerName="maas-api"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.697762    4577 cpu_manager.go:414] "containerMap: removing container" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.697770    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.697829    4577 cpu_manager.go:414] "containerMap: removing container" podUID="274b1218-9bc3-4101-b090-60f03462c303" containerName="payload-processing"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.697836    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="274b1218-9bc3-4101-b090-60f03462c303" containerName="payload-processing"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.698032    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="274b1218-9bc3-4101-b090-60f03462c303" containerName="payload-processing"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.698057    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.698069    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4811ec2-ed4f-4afa-9e94-a6e33b9624b8" containerName="maas-api"
Oct 30 01:08:32.698905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.698081    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy"
Oct 30 01:08:32.706303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.705737    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.709205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.709182    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"]
Oct 30 01:08:32.777584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.777552    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d3c1fa-openshift-default-dockercfg-crp66\""
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878078    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878164    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878228    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878295    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878332    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878355    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878386    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ql8\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878414    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.885882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.878471    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:32.963082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:32.962413    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:33.080361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.080320    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.082140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.082114    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.097869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.083163    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.098462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.098434    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.098654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.098637    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.098775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.098759    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.098956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.098939    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ql8\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.099103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.099087    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.099269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.099252    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.099902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.082949    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.113289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.107242    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.113485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.081723    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.113586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.112552    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.113682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.087361    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.154722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.154681    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.182868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.178801    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:33.218566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.203175    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.219742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.219202    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.221058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.220608    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ql8\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8\") pod \"e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") " pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.326989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.326955    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:33.614990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.614953    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:33.712601 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:33.712569    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf49c3e7_04b3_4ca8_8e90_4dcd078bfbf6.slice/crio-9720eb454b6d894dd30ba57eb78bf0cd416c66fa5796dfa5d8e0499785c8efdf WatchSource:0}: Error finding container 9720eb454b6d894dd30ba57eb78bf0cd416c66fa5796dfa5d8e0499785c8efdf: Status 404 returned error can't find the container with id 9720eb454b6d894dd30ba57eb78bf0cd416c66fa5796dfa5d8e0499785c8efdf
Oct 30 01:08:33.712827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.712806    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"]
Oct 30 01:08:33.715157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.715090    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:33.715265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.715210    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:33.715265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:33.715257    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:34.232815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:34.232774    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" event={"ID":"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6","Type":"ContainerStarted","Data":"89e78379fa5969b5f067c26c1b09f7cb06513e5b7cc1b43ac9faec57aeb93acd"}
Oct 30 01:08:34.233339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:34.232834    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" event={"ID":"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6","Type":"ContainerStarted","Data":"9720eb454b6d894dd30ba57eb78bf0cd416c66fa5796dfa5d8e0499785c8efdf"}
Oct 30 01:08:34.314909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:34.314836    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podStartSLOduration=2.314818702 podStartE2EDuration="2.314818702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:34.300258731 +0000 UTC m=+1622.329568963" watchObservedRunningTime="2026-10-30 01:08:34.314818702 +0000 UTC m=+1622.344128934"
Oct 30 01:08:34.327337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:34.327312    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:34.332932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:34.332912    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:35.283368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:35.283337    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:35.284607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:35.284585    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:08:35.636895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:35.636784    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:36.805120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:36.805089    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:37.034683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:37.034632    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:37.934660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:37.934626    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:38.283016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:38.282975    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:08:38.283223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:38.283032    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:08:39.624716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.624676    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:39.893232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.893142    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"]
Oct 30 01:08:39.894604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.894559    4577 cpu_manager.go:414] "containerMap: removing container" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy"
Oct 30 01:08:39.894604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.894604    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="421db701-5ef1-4a24-8fc6-342e88a351bf" containerName="istio-proxy"
Oct 30 01:08:39.903730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.903695    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:39.922715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.922688    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d3c1fa\""
Oct 30 01:08:39.922872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.922775    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d3c1fa-dockercfg-4vw8j\""
Oct 30 01:08:39.929255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.929226    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:39.944097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:39.944067    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"]
Oct 30 01:08:40.138963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.081225    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.138963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.081380    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxjxw\" (UniqueName: \"kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.185572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.183008    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.185572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.183211    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxjxw\" (UniqueName: \"kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.187583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.187452    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.219217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.219142    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:40.232001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.231192    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxjxw\" (UniqueName: \"kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.367274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.366522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:40.526045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.526010    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:40.863659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.863622    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"]
Oct 30 01:08:40.865263 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:40.865212    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76db7b4_27aa_4be2_9ccc_6d07b3ab1505.slice/crio-42c6fa7b44e7797bea895dc43cb075ac00a8e7461faac53c3110b04cb55f47d2 WatchSource:0}: Error finding container 42c6fa7b44e7797bea895dc43cb075ac00a8e7461faac53c3110b04cb55f47d2: Status 404 returned error can't find the container with id 42c6fa7b44e7797bea895dc43cb075ac00a8e7461faac53c3110b04cb55f47d2
Oct 30 01:08:40.993260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.993231    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"]
Oct 30 01:08:40.993488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:40.993465    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy" containerID="cri-o://c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d" gracePeriod=30
Oct 30 01:08:41.082913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.082883    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:41.426426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.426396    4577 generic.go:356] "Generic (PLEG): container finished" podID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerID="efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b" exitCode=0
Oct 30 01:08:41.426606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.426469    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" event={"ID":"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0","Type":"ContainerDied","Data":"efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"}
Oct 30 01:08:41.427022 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.426989    4577 scope.go:122] "RemoveContainer" containerID="efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"
Oct 30 01:08:41.428522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.428488    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x" event={"ID":"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505","Type":"ContainerStarted","Data":"607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796"}
Oct 30 01:08:41.428629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.428547    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x" event={"ID":"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505","Type":"ContainerStarted","Data":"42c6fa7b44e7797bea895dc43cb075ac00a8e7461faac53c3110b04cb55f47d2"}
Oct 30 01:08:41.428629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.428615    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:41.428806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.428782    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:41.428893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.428872    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:41.428942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.428906    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:41.481346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.481318    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:41.542370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.542311    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x" podStartSLOduration=2.542291195 podStartE2EDuration="2.542291195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:41.537201706 +0000 UTC m=+1629.566511947" watchObservedRunningTime="2026-10-30 01:08:41.542291195 +0000 UTC m=+1629.571601435"
Oct 30 01:08:41.925164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.925133    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"]
Oct 30 01:08:41.935792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.935771    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:41.957551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.957522    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-50d4ea-openshift-default-dockercfg-cv6c9\""
Oct 30 01:08:41.966907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:41.965568    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"]
Oct 30 01:08:42.192633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.192549    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:42.193205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193170    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193223    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193394    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193472    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193502    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2ltn\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193626    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193675    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193704    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.193770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.193727    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.303806    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.303879    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2ltn\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304010    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304049    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304082    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304107    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304183    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304215    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.304424    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.308997    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.309627    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.309879    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.314870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.310502    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.315743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.315441    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.360917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.351518    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.360917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.352175    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2ltn\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.360917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.352658    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.367401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.367371    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo\") pod \"e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") " pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.438342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.438307    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" event={"ID":"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0","Type":"ContainerStarted","Data":"1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8"}
Oct 30 01:08:42.517604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.517559    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:42.564172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.564122    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:42.913204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.913176    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"]
Oct 30 01:08:42.915066 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:42.915036    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd9de74_415f_46de_849d_d846276afdb7.slice/crio-2d9dcc6f27e56e9f9b2270fed6c27f2bd995b1a6434c32b82e20ba5de38162c3 WatchSource:0}: Error finding container 2d9dcc6f27e56e9f9b2270fed6c27f2bd995b1a6434c32b82e20ba5de38162c3: Status 404 returned error can't find the container with id 2d9dcc6f27e56e9f9b2270fed6c27f2bd995b1a6434c32b82e20ba5de38162c3
Oct 30 01:08:42.919152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.919117    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:42.919254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.919185    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:42.919254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:42.919215    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:08:43.023837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.023811    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:08:43.024191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.024092    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 01:08:43.024191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.024132    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 01:08:43.471870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.470374    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" event={"ID":"0dd9de74-415f-46de-849d-d846276afdb7","Type":"ContainerStarted","Data":"838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"}
Oct 30 01:08:43.471870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.470435    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" event={"ID":"0dd9de74-415f-46de-849d-d846276afdb7","Type":"ContainerStarted","Data":"2d9dcc6f27e56e9f9b2270fed6c27f2bd995b1a6434c32b82e20ba5de38162c3"}
Oct 30 01:08:43.471870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.471137    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:43.566991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.566924    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:43.580301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:43.580238    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podStartSLOduration=2.580218993 podStartE2EDuration="2.580218993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:43.560684751 +0000 UTC m=+1631.589994996" watchObservedRunningTime="2026-10-30 01:08:43.580218993 +0000 UTC m=+1631.609529226"
Oct 30 01:08:44.024031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:44.024001    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 01:08:44.024502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:44.024051    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 01:08:44.577720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:44.577691    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:44.988899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:44.988642    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:45.024087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:45.024060    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 01:08:45.024472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:45.024120    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 01:08:45.187569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:45.184463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:45.572869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:45.560524    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:45.572869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:45.571370    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:08:45.739161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:45.738476    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:46.023264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.023238    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 01:08:46.023347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.023291    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 01:08:46.321929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.321902    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:08:46.420485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420455    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.420696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420524    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.420696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420573    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.420696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420613    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.420696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420679    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420725    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420758    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420816    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420943    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.420979    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421008    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421025    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy\") pod \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\" (UID: \"6adfe7e6-3179-4162-b94c-566f2c3ffc96\") "
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421187    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:46.421356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421337    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:46.423410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421634    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.423410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421655    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.423410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421668    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.423410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421681    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.423410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.421693    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.425109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.425087    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:08:46.425567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.425494    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "kube-api-access-b4gxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:08:46.430064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.425784    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:08:46.430064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.426791    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy" pod "6adfe7e6-3179-4162-b94c-566f2c3ffc96" (UID: "6adfe7e6-3179-4162-b94c-566f2c3ffc96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:08:46.531700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.531655    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.531700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.531692    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.532083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.531706    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4gxc\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-kube-api-access-b4gxc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.532083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.531764    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adfe7e6-3179-4162-b94c-566f2c3ffc96-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:08:46.572599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.558914    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:46.582504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.582481    4577 generic.go:356] "Generic (PLEG): container finished" podID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerID="c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d" exitCode=0
Oct 30 01:08:46.582677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.582655    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" event={"ID":"6adfe7e6-3179-4162-b94c-566f2c3ffc96","Type":"ContainerDied","Data":"c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d"}
Oct 30 01:08:46.582762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.582749    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf" event={"ID":"6adfe7e6-3179-4162-b94c-566f2c3ffc96","Type":"ContainerDied","Data":"45f7c4e59fbdeb62ca3f39c19829b6635d2cafceb6526ca026d4ee9b03727bbc"}
Oct 30 01:08:46.582824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.582786    4577 scope.go:122] "RemoveContainer" containerID="c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d"
Oct 30 01:08:46.584505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.582982    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"
Oct 30 01:08:46.600923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.600905    4577 scope.go:122] "RemoveContainer" containerID="3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"
Oct 30 01:08:46.610580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.610548    4577 scope.go:122] "RemoveContainer" containerID="c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d"
Oct 30 01:08:46.610875 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:46.610832    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d\": container with ID starting with c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d not found: ID does not exist" containerID="c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d"
Oct 30 01:08:46.610966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.610888    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d"} err="failed to get container status \"c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d\": rpc error: code = NotFound desc = could not find container \"c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d\": container with ID starting with c08f97567798a6550bb9afc0553bfc6f46d39364f1afb12cada39c173f74fe0d not found: ID does not exist"
Oct 30 01:08:46.610966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.610911    4577 scope.go:122] "RemoveContainer" containerID="3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"
Oct 30 01:08:46.611215 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:46.611197    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2\": container with ID starting with 3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2 not found: ID does not exist" containerID="3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"
Oct 30 01:08:46.611271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.611235    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2"} err="failed to get container status \"3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2\": rpc error: code = NotFound desc = could not find container \"3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2\": container with ID starting with 3550d2814f78645b0c0676fc9dc144bade5de97f7f818842753a2a035dbebed2 not found: ID does not exist"
Oct 30 01:08:46.677557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.677508    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"]
Oct 30 01:08:46.682272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.682249    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-11fa15-openshift-default-6c69b46957-n8pzf"]
Oct 30 01:08:46.777591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:46.777535    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:47.023355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:47.023317    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 01:08:47.023546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:47.023371    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 01:08:48.113637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.113605    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" path="/var/lib/kubelet/pods/6adfe7e6-3179-4162-b94c-566f2c3ffc96/volumes"
Oct 30 01:08:48.284024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.283921    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:48.303683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.303643    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:08:48.304778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.304693    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:08:48.306317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.306191    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:08:48.469315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.469219    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"]
Oct 30 01:08:48.470520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.470494    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy"
Oct 30 01:08:48.470681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.470666    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy"
Oct 30 01:08:48.470791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.470779    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy"
Oct 30 01:08:48.470923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.470908    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy"
Oct 30 01:08:48.471293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.471275    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy"
Oct 30 01:08:48.471402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.471390    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6adfe7e6-3179-4162-b94c-566f2c3ffc96" containerName="istio-proxy"
Oct 30 01:08:48.476398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.476379    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.485611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.485419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:48.498021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.497998    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-50d4ea\""
Oct 30 01:08:48.498165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.498027    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"]
Oct 30 01:08:48.498303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.498155    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-50d4ea-dockercfg-7bgch\""
Oct 30 01:08:48.541779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.541752    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"]
Oct 30 01:08:48.545080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.545060    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.572998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.571805    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztn6q\" (UniqueName: \"kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q\") pod \"payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.572998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.572321    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.613879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.584121    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"]
Oct 30 01:08:48.676900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.676838    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztn6q\" (UniqueName: \"kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q\") pod \"payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.685868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.677242    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlhk9\" (UniqueName: \"kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9\") pod \"payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.685868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.677344    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.685868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.677455    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.685868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.678403    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.747880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.746582    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztn6q\" (UniqueName: \"kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q\") pod \"payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.786872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.780149    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.786872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.780484    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlhk9\" (UniqueName: \"kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9\") pod \"payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.786872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.783466    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.802878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.802169    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:48.811064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.810479    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlhk9\" (UniqueName: \"kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9\") pod \"payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.883737 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.883481    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:48.931831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:48.931792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:49.171403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.171378    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"]
Oct 30 01:08:49.173215 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:49.173173    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb372fed_4f64_4844_bec2_df11368f47f9.slice/crio-e32a815b2a04aff3fa9b1a1c39c2448b75e9d0fe03d9c6c844402b903f161c52 WatchSource:0}: Error finding container e32a815b2a04aff3fa9b1a1c39c2448b75e9d0fe03d9c6c844402b903f161c52: Status 404 returned error can't find the container with id e32a815b2a04aff3fa9b1a1c39c2448b75e9d0fe03d9c6c844402b903f161c52
Oct 30 01:08:49.215276 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:49.215246    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a99c88_90a1_4c56_ab07_64ed4e242094.slice/crio-991724afdb9ef6f8ad45d66ba76ef6c5887d8c59f5ee5aee5256123e02f99033 WatchSource:0}: Error finding container 991724afdb9ef6f8ad45d66ba76ef6c5887d8c59f5ee5aee5256123e02f99033: Status 404 returned error can't find the container with id 991724afdb9ef6f8ad45d66ba76ef6c5887d8c59f5ee5aee5256123e02f99033
Oct 30 01:08:49.216312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.216286    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"]
Oct 30 01:08:49.764725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.764685    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph" event={"ID":"bb372fed-4f64-4844-bec2-df11368f47f9","Type":"ContainerStarted","Data":"5f291fa2fac05b43af98591911923a57e2056d6b73b8665ae1c9b0d8928a5566"}
Oct 30 01:08:49.764978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.764789    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph" event={"ID":"bb372fed-4f64-4844-bec2-df11368f47f9","Type":"ContainerStarted","Data":"e32a815b2a04aff3fa9b1a1c39c2448b75e9d0fe03d9c6c844402b903f161c52"}
Oct 30 01:08:49.764978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.764825    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:08:49.766729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.766700    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh" event={"ID":"94a99c88-90a1-4c56-ab07-64ed4e242094","Type":"ContainerStarted","Data":"a4b3b9e0b61fcd9dc797c6754075be4f7c7ccafca8bce74adaf68052f8771797"}
Oct 30 01:08:49.766871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.766755    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh" event={"ID":"94a99c88-90a1-4c56-ab07-64ed4e242094","Type":"ContainerStarted","Data":"991724afdb9ef6f8ad45d66ba76ef6c5887d8c59f5ee5aee5256123e02f99033"}
Oct 30 01:08:49.766998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.766976    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:08:49.814776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.814718    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph" podStartSLOduration=1.8147049210000001 podStartE2EDuration="1.814704921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:49.812170195 +0000 UTC m=+1637.841480427" watchObservedRunningTime="2026-10-30 01:08:49.814704921 +0000 UTC m=+1637.844015194"
Oct 30 01:08:49.868713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:49.868660    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh" podStartSLOduration=1.868647615 podStartE2EDuration="1.868647615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:49.86244692 +0000 UTC m=+1637.891757155" watchObservedRunningTime="2026-10-30 01:08:49.868647615 +0000 UTC m=+1637.897957846"
Oct 30 01:08:50.643476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:50.643448    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:52.442111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:52.442086    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:08:52.591502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:52.591469    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:55.041587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:55.041557    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:56.968874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:56.968827    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:57.038866 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:08:57.037283    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:08:58.313810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.313778    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:58.507403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.507363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:58.769716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.769679    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"]
Oct 30 01:08:58.795144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.795116    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:58.799559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.799526    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"]
Oct 30 01:08:58.915888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.915809    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-429c67-dockercfg-lph86\""
Oct 30 01:08:58.916221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.916178    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-429c67\""
Oct 30 01:08:58.951450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.951383    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"]
Oct 30 01:08:58.955727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.955702    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:58.970037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.970014    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"]
Oct 30 01:08:58.994148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.994112    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwpkf\" (UniqueName: \"kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf\") pod \"payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:58.994335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.994183    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:58.994335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.994238    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:58.994335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:58.994284    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frm2l\" (UniqueName: \"kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l\") pod \"payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.095325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.095228    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwpkf\" (UniqueName: \"kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf\") pod \"payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:59.095325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.095325    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:59.095582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.095355    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.095582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.095397    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frm2l\" (UniqueName: \"kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l\") pod \"payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.096165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.096144    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.096215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.096162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:59.121574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.121548    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frm2l\" (UniqueName: \"kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l\") pod \"payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.121974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.121950    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwpkf\" (UniqueName: \"kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf\") pod \"payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:59.225325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.225298    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.277940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.277892    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:59.555585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.554456    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"]
Oct 30 01:08:59.555585 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:59.554839    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2b51c6_2d0e_4d6a_b1c8_93dd4a9524eb.slice/crio-aa2c43e08b23172ab7a67d547681efcc7108c30aa257a49740e4661df5ba2a01 WatchSource:0}: Error finding container aa2c43e08b23172ab7a67d547681efcc7108c30aa257a49740e4661df5ba2a01: Status 404 returned error can't find the container with id aa2c43e08b23172ab7a67d547681efcc7108c30aa257a49740e4661df5ba2a01
Oct 30 01:08:59.612371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.612347    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"]
Oct 30 01:08:59.615365 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:08:59.615336    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87e08bb_d73e_4702_8d8e_92be5ae8190f.slice/crio-cdbfbf976d04d8feee73eefd3361c6b401111117e49c7943640814548c1a6d1a WatchSource:0}: Error finding container cdbfbf976d04d8feee73eefd3361c6b401111117e49c7943640814548c1a6d1a: Status 404 returned error can't find the container with id cdbfbf976d04d8feee73eefd3361c6b401111117e49c7943640814548c1a6d1a
Oct 30 01:08:59.773214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.770838    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:08:59.929352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.929261    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc" event={"ID":"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb","Type":"ContainerStarted","Data":"f076d1d52df7167d46da8f74fd0d16ac0ae6a666312aff27fbc8742cc898516b"}
Oct 30 01:08:59.929352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.929323    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc" event={"ID":"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb","Type":"ContainerStarted","Data":"aa2c43e08b23172ab7a67d547681efcc7108c30aa257a49740e4661df5ba2a01"}
Oct 30 01:08:59.929352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.929351    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:08:59.931042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.931016    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz" event={"ID":"f87e08bb-d73e-4702-8d8e-92be5ae8190f","Type":"ContainerStarted","Data":"66ba403cecf598b25be9cb79bfd8cb22a664fde1db4182d5d0858276620cab95"}
Oct 30 01:08:59.931152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.931058    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz" event={"ID":"f87e08bb-d73e-4702-8d8e-92be5ae8190f","Type":"ContainerStarted","Data":"cdbfbf976d04d8feee73eefd3361c6b401111117e49c7943640814548c1a6d1a"}
Oct 30 01:08:59.931152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.931084    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:08:59.993242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:08:59.993188    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc" podStartSLOduration=1.993169436 podStartE2EDuration="1.993169436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:59.992295162 +0000 UTC m=+1648.021605398" watchObservedRunningTime="2026-10-30 01:08:59.993169436 +0000 UTC m=+1648.022479672"
Oct 30 01:09:00.041939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:00.041891    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz" podStartSLOduration=2.041877762 podStartE2EDuration="2.041877762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:00.038117099 +0000 UTC m=+1648.067427330" watchObservedRunningTime="2026-10-30 01:09:00.041877762 +0000 UTC m=+1648.071187992"
Oct 30 01:09:00.779825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:00.779793    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:09:00.780256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:00.780226    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:09:01.851585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:01.846928    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"]
Oct 30 01:09:01.851585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:01.847176    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" podUID="e6b5ffd5-771a-46b6-8806-4fead8f98d6e" containerName="maas-api" containerID="cri-o://f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9" gracePeriod=30
Oct 30 01:09:01.949036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:01.949002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:02.121063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.121009    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:09:02.140449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.140412    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\") pod \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") "
Oct 30 01:09:02.140661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.140645    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs\") pod \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") "
Oct 30 01:09:02.140742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.140675    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn\" (UniqueName: \"kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn\") pod \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") "
Oct 30 01:09:02.140742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.140713    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca\") pod \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\" (UID: \"e6b5ffd5-771a-46b6-8806-4fead8f98d6e\") "
Oct 30 01:09:02.145371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.141539    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca" pod "e6b5ffd5-771a-46b6-8806-4fead8f98d6e" (UID: "e6b5ffd5-771a-46b6-8806-4fead8f98d6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:02.164975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.163786    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls" pod "e6b5ffd5-771a-46b6-8806-4fead8f98d6e" (UID: "e6b5ffd5-771a-46b6-8806-4fead8f98d6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:02.322122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.297080    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:02.322122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.297121    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:02.322122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.297151    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"]
Oct 30 01:09:02.322122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.297390    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6" podUID="95e1f7c9-a823-41cc-9417-a81e702f3e3f" containerName="payload-processing" containerID="cri-o://d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d" gracePeriod=30
Oct 30 01:09:02.322122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.318197    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn" pod "e6b5ffd5-771a-46b6-8806-4fead8f98d6e" (UID: "e6b5ffd5-771a-46b6-8806-4fead8f98d6e"). InnerVolumeSpecName "kube-api-access-6mtbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:02.322122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.320295    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs" pod "e6b5ffd5-771a-46b6-8806-4fead8f98d6e" (UID: "e6b5ffd5-771a-46b6-8806-4fead8f98d6e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:02.398482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.398410    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:02.398482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.398437    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mtbn\" (UniqueName: \"kubernetes.io/projected/e6b5ffd5-771a-46b6-8806-4fead8f98d6e-kube-api-access-6mtbn\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:02.489791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.489749    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:02.603609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.603577    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:09:02.698263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.693784    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:02.717874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.711425    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume\") pod \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") "
Oct 30 01:09:02.717874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.711545    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv\" (UniqueName: \"kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv\") pod \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\" (UID: \"95e1f7c9-a823-41cc-9417-a81e702f3e3f\") "
Oct 30 01:09:02.743265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.743200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume" pod "95e1f7c9-a823-41cc-9417-a81e702f3e3f" (UID: "95e1f7c9-a823-41cc-9417-a81e702f3e3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:02.832930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.816456    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95e1f7c9-a823-41cc-9417-a81e702f3e3f-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:02.840065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.839588    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv" pod "95e1f7c9-a823-41cc-9417-a81e702f3e3f" (UID: "95e1f7c9-a823-41cc-9417-a81e702f3e3f"). InnerVolumeSpecName "kube-api-access-s74gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:02.874827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.874743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:02.909232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.906880    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"]
Oct 30 01:09:02.914488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.914459    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e6b5ffd5-771a-46b6-8806-4fead8f98d6e" containerName="maas-api"
Oct 30 01:09:02.924174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.924146    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6b5ffd5-771a-46b6-8806-4fead8f98d6e" containerName="maas-api"
Oct 30 01:09:02.924385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.924370    4577 cpu_manager.go:414] "containerMap: removing container" podUID="95e1f7c9-a823-41cc-9417-a81e702f3e3f" containerName="payload-processing"
Oct 30 01:09:02.924502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.924489    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95e1f7c9-a823-41cc-9417-a81e702f3e3f" containerName="payload-processing"
Oct 30 01:09:02.924882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.924837    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6b5ffd5-771a-46b6-8806-4fead8f98d6e" containerName="maas-api"
Oct 30 01:09:02.930971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.930949    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="95e1f7c9-a823-41cc-9417-a81e702f3e3f" containerName="payload-processing"
Oct 30 01:09:02.937583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.937552    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s74gv\" (UniqueName: \"kubernetes.io/projected/95e1f7c9-a823-41cc-9417-a81e702f3e3f-kube-api-access-s74gv\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:02.961332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.961236    4577 generic.go:356] "Generic (PLEG): container finished" podID="e6b5ffd5-771a-46b6-8806-4fead8f98d6e" containerID="f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9" exitCode=0
Oct 30 01:09:02.961490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.961428    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"
Oct 30 01:09:02.966280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.966252    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:02.967338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.967312    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"]
Oct 30 01:09:02.967438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.967385    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" event={"ID":"e6b5ffd5-771a-46b6-8806-4fead8f98d6e","Type":"ContainerDied","Data":"f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9"}
Oct 30 01:09:02.967517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.967441    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr" event={"ID":"e6b5ffd5-771a-46b6-8806-4fead8f98d6e","Type":"ContainerDied","Data":"2ccde1166a485ed6e7ce19eda7f7984400b5a53b650003d70f1507d202ba4d82"}
Oct 30 01:09:02.967583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.967472    4577 scope.go:122] "RemoveContainer" containerID="f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9"
Oct 30 01:09:02.978937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.978907    4577 generic.go:356] "Generic (PLEG): container finished" podID="95e1f7c9-a823-41cc-9417-a81e702f3e3f" containerID="d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d" exitCode=0
Oct 30 01:09:02.979086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.979002    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6" event={"ID":"95e1f7c9-a823-41cc-9417-a81e702f3e3f","Type":"ContainerDied","Data":"d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d"}
Oct 30 01:09:02.979086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.979026    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"
Oct 30 01:09:02.979086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.979045    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6" event={"ID":"95e1f7c9-a823-41cc-9417-a81e702f3e3f","Type":"ContainerDied","Data":"bafd8978d17ccbbe65c6465ebe3c8ac7ccca1caaab380a94ff0aac55a284ac31"}
Oct 30 01:09:02.988008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:02.987986    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d3c1fa-serving-cert\""
Oct 30 01:09:03.002305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.002284    4577 scope.go:122] "RemoveContainer" containerID="f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9"
Oct 30 01:09:03.002640 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:03.002621    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9\": container with ID starting with f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9 not found: ID does not exist" containerID="f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9"
Oct 30 01:09:03.002693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.002649    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9"} err="failed to get container status \"f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9\": rpc error: code = NotFound desc = could not find container \"f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9\": container with ID starting with f60e24a29ad5751b2625c05c88663c9082c1a9d547c599a26ae857ac3e7bf5e9 not found: ID does not exist"
Oct 30 01:09:03.002693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.002667    4577 scope.go:122] "RemoveContainer" containerID="d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d"
Oct 30 01:09:03.008472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.008449    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:03.012553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.012534    4577 scope.go:122] "RemoveContainer" containerID="d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d"
Oct 30 01:09:03.012874 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:03.012826    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d\": container with ID starting with d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d not found: ID does not exist" containerID="d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d"
Oct 30 01:09:03.012956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.012868    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d"} err="failed to get container status \"d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d\": rpc error: code = NotFound desc = could not find container \"d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d\": container with ID starting with d103b180afcf0ff61e06de23227d79516ecbe43b487b0bec2f525298a4e9095d not found: ID does not exist"
Oct 30 01:09:03.166939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.155191    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.166939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.155512    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c28jl\" (UniqueName: \"kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.166939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.155589    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.166939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.155794    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.183077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.176744    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"]
Oct 30 01:09:03.185299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.183592    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-838fd5-665487df5f-2jctr"]
Oct 30 01:09:03.185973 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.185942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:03.188332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.188308    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"]
Oct 30 01:09:03.191981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.191956    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-838fd5-8f765b7bd-szzw6"]
Oct 30 01:09:03.257384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.257356    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c28jl\" (UniqueName: \"kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.257543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.257407    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.257543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.257483    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.257695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.257560    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.258340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.258315    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.260165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.260143    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.260298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.260278    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.286556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.286526    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c28jl\" (UniqueName: \"kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl\") pod \"maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.297326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.297302    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:03.374517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.374482    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:03.551693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.551615    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:03.609228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.609201    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"]
Oct 30 01:09:03.609373 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:09:03.609277    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44183be9_c26d_4f52_bc5f_f7a65dfadf14.slice/crio-d8b06db6281f0fc85ac08ddf94b0738053c446ccab86717a3f736388228df5a9 WatchSource:0}: Error finding container d8b06db6281f0fc85ac08ddf94b0738053c446ccab86717a3f736388228df5a9: Status 404 returned error can't find the container with id d8b06db6281f0fc85ac08ddf94b0738053c446ccab86717a3f736388228df5a9
Oct 30 01:09:03.990924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:03.990887    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n" event={"ID":"44183be9-c26d-4f52-bc5f-f7a65dfadf14","Type":"ContainerStarted","Data":"d8b06db6281f0fc85ac08ddf94b0738053c446ccab86717a3f736388228df5a9"}
Oct 30 01:09:04.114053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:04.114022    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95e1f7c9-a823-41cc-9417-a81e702f3e3f" path="/var/lib/kubelet/pods/95e1f7c9-a823-41cc-9417-a81e702f3e3f/volumes"
Oct 30 01:09:04.114609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:04.114586    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6b5ffd5-771a-46b6-8806-4fead8f98d6e" path="/var/lib/kubelet/pods/e6b5ffd5-771a-46b6-8806-4fead8f98d6e/volumes"
Oct 30 01:09:05.003168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:05.003128    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n" event={"ID":"44183be9-c26d-4f52-bc5f-f7a65dfadf14","Type":"ContainerStarted","Data":"797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a"}
Oct 30 01:09:05.003583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:05.003225    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:05.052429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:05.052386    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n" podStartSLOduration=2.655572212 podStartE2EDuration="3.052373551s" totalImagesPullingTime="396.801339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:05.05135136 +0000 UTC m=+1653.080661591" watchObservedRunningTime="2026-10-30 01:09:05.052373551 +0000 UTC m=+1653.081683810"
Oct 30 01:09:05.418163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:05.418077    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:07.087718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:07.087683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:08.909104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:08.909070    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:10.551419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:10.547360    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:10.699596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:10.699567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:10.884504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:10.884428    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"]
Oct 30 01:09:10.884925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:10.884898    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" containerID="cri-o://677b0e21da126cf5914dd3959f8658590499d9ecf7e1fb3143b609a9b495f59d" gracePeriod=30
Oct 30 01:09:11.001947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:10.982986    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:09:11.017902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:11.017718    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:11.025923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:11.024910    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:09:11.208919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:11.208774    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:09:11.550295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:11.547248    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:13.331411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.331326    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"]
Oct 30 01:09:13.337201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.337177    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.367580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.367550    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-50d4ea-serving-cert\""
Oct 30 01:09:13.374419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.374392    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"]
Oct 30 01:09:13.425504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.425475    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:13.496975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.496940    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.497157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.497081    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.497236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.497222    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.497361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.497341    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.548618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.548585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:13.601920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.599025    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.601920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.599128    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.601920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.599194    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.601920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.599315    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.615926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.607985    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.616186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.609211    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.623138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.623106    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.642282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.642256    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m\") pod \"maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.650997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.650973    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:13.698298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.698244    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:13.759643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.756331    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:09:13.759643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.756395    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:09:13.978520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:13.978486    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"]
Oct 30 01:09:14.099669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:14.099613    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:14.113724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:14.113687    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj" event={"ID":"9ee034c4-a2fd-43f1-84da-e10cf534d5b6","Type":"ContainerStarted","Data":"a6d8c5be07fba261f56a479a9ff5dbd341c323815e53630c3de7f13fa6447c8d"}
Oct 30 01:09:14.257869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:14.252969    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:15.135530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:15.135494    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj" event={"ID":"9ee034c4-a2fd-43f1-84da-e10cf534d5b6","Type":"ContainerStarted","Data":"30e9694bf9ee4b751c3c72fdcd594c33fbb3b692627067d012203d38e158c796"}
Oct 30 01:09:15.136042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:15.135595    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:15.184655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:15.184597    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj" podStartSLOduration=1.62961216 podStartE2EDuration="2.184582296s" totalImagesPullingTime="554.970136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:15.182538877 +0000 UTC m=+1663.211849131" watchObservedRunningTime="2026-10-30 01:09:15.184582296 +0000 UTC m=+1663.213892577"
Oct 30 01:09:15.602891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:15.602787    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:16.238268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.238231    4577 generic.go:356] "Generic (PLEG): container finished" podID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerID="677b0e21da126cf5914dd3959f8658590499d9ecf7e1fb3143b609a9b495f59d" exitCode=0
Oct 30 01:09:16.242615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.238625    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" event={"ID":"47dcdb1c-2440-4ffc-aecd-8f7dae131d12","Type":"ContainerDied","Data":"677b0e21da126cf5914dd3959f8658590499d9ecf7e1fb3143b609a9b495f59d"}
Oct 30 01:09:16.242772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.242761    4577 scope.go:122] "RemoveContainer" containerID="6a2a86556eba28fa9e9b81775485d9b45ec369a01869bca446973977784aa3ff"
Oct 30 01:09:16.267792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.267767    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:09:16.371553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371517    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.371749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371609    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.371749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371698    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.371749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371743    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.371961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371876    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.371961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371871    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:16.371961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.371907    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.372112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.372005    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.372112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.372066    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.372230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.372119    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs\") pod \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\" (UID: \"47dcdb1c-2440-4ffc-aecd-8f7dae131d12\") "
Oct 30 01:09:16.372230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.372204    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:16.372924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.372884    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.372924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.372912    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.374537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.374182    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:16.374764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.374714    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:16.375010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.374970    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:16.375990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.375900    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:09:16.378156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.378101    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "kube-api-access-k567k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:16.378718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.378322    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:16.380247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.380222    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy" pod "47dcdb1c-2440-4ffc-aecd-8f7dae131d12" (UID: "47dcdb1c-2440-4ffc-aecd-8f7dae131d12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:16.474449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474418    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.474449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474444    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k567k\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-kube-api-access-k567k\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.474449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474456    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.474695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474464    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.474695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474473    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.474695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474481    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:16.474695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:16.474489    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47dcdb1c-2440-4ffc-aecd-8f7dae131d12-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:17.248060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:17.248026    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz" event={"ID":"47dcdb1c-2440-4ffc-aecd-8f7dae131d12","Type":"ContainerDied","Data":"d07e4221aed1705189f3e8dc20470a26ad72c6859e9764d97d41678308940e91"}
Oct 30 01:09:17.248528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:17.248081    4577 scope.go:122] "RemoveContainer" containerID="677b0e21da126cf5914dd3959f8658590499d9ecf7e1fb3143b609a9b495f59d"
Oct 30 01:09:17.248528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:17.248040    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"
Oct 30 01:09:17.367083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:17.367051    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"]
Oct 30 01:09:17.370656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:17.370629    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-838fd5-openshift-default-5f99fdcb8f-dshfz"]
Oct 30 01:09:17.624158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:17.624068    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:18.116154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:18.116123    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" path="/var/lib/kubelet/pods/47dcdb1c-2440-4ffc-aecd-8f7dae131d12/volumes"
Oct 30 01:09:18.465538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:18.465450    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:18.594590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:18.594557    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:19.613504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:19.613465    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:20.187337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:20.187301    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:20.317699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:20.317663    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:21.245813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.245781    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"]
Oct 30 01:09:21.246935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.246880    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api" containerID="cri-o://6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099" gracePeriod=30
Oct 30 01:09:21.266423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.266396    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:09:21.302426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.302394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:21.416210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.416181    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"]
Oct 30 01:09:21.416791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.416763    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8" podUID="4e2b578c-c2df-41db-92a2-406a099c5638" containerName="payload-processing" containerID="cri-o://8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99" gracePeriod=30
Oct 30 01:09:21.430243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.430191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:21.434323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.434300    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"]
Oct 30 01:09:21.434747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.434725    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj" podUID="3d1e47c1-33b1-4153-ae91-b44a72174b1a" containerName="payload-pre-processing" containerID="cri-o://3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c" gracePeriod=30
Oct 30 01:09:21.554380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.550253    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:21.654394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.653271    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:09:21.898573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.773986    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q\" (UniqueName: \"kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q\") pod \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") "
Oct 30 01:09:21.898573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.870512    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs\") pod \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") "
Oct 30 01:09:21.898573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.870790    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\") pod \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") "
Oct 30 01:09:21.898573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.870941    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca\") pod \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\" (UID: \"06ef12f9-f0fb-4e0e-b131-7ae244af3855\") "
Oct 30 01:09:21.898573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.837489    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q" pod "06ef12f9-f0fb-4e0e-b131-7ae244af3855" (UID: "06ef12f9-f0fb-4e0e-b131-7ae244af3855"). InnerVolumeSpecName "kube-api-access-9mt7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:21.898573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.883483    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca" pod "06ef12f9-f0fb-4e0e-b131-7ae244af3855" (UID: "06ef12f9-f0fb-4e0e-b131-7ae244af3855"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:21.902307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.901653    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs" pod "06ef12f9-f0fb-4e0e-b131-7ae244af3855" (UID: "06ef12f9-f0fb-4e0e-b131-7ae244af3855"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:21.903467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.902718    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls" pod "06ef12f9-f0fb-4e0e-b131-7ae244af3855" (UID: "06ef12f9-f0fb-4e0e-b131-7ae244af3855"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:21.951121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.945274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:21.991520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.991160    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mt7q\" (UniqueName: \"kubernetes.io/projected/06ef12f9-f0fb-4e0e-b131-7ae244af3855-kube-api-access-9mt7q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:21.991520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.991216    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:21.991520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.991232    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06ef12f9-f0fb-4e0e-b131-7ae244af3855-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:21.991520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:21.991247    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06ef12f9-f0fb-4e0e-b131-7ae244af3855-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:22.017812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.017364    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:09:22.076751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.076721    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:22.080158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.080137    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:09:22.095318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.094366    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume\") pod \"4e2b578c-c2df-41db-92a2-406a099c5638\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") "
Oct 30 01:09:22.095318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.094624    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc\" (UniqueName: \"kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc\") pod \"4e2b578c-c2df-41db-92a2-406a099c5638\" (UID: \"4e2b578c-c2df-41db-92a2-406a099c5638\") "
Oct 30 01:09:22.095318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.095274    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume" pod "4e2b578c-c2df-41db-92a2-406a099c5638" (UID: "4e2b578c-c2df-41db-92a2-406a099c5638"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:22.095556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.095473    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b578c-c2df-41db-92a2-406a099c5638-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:22.098319 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.098291    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc" pod "4e2b578c-c2df-41db-92a2-406a099c5638" (UID: "4e2b578c-c2df-41db-92a2-406a099c5638"). InnerVolumeSpecName "kube-api-access-wqslc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:22.196933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.196749    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:22.197643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.197611    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6\" (UniqueName: \"kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6\") pod \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") "
Oct 30 01:09:22.204594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.204552    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume\") pod \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\" (UID: \"3d1e47c1-33b1-4153-ae91-b44a72174b1a\") "
Oct 30 01:09:22.208653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.205720    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqslc\" (UniqueName: \"kubernetes.io/projected/4e2b578c-c2df-41db-92a2-406a099c5638-kube-api-access-wqslc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:22.214988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.209460    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume" pod "3d1e47c1-33b1-4153-ae91-b44a72174b1a" (UID: "3d1e47c1-33b1-4153-ae91-b44a72174b1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:22.249278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.249230    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6" pod "3d1e47c1-33b1-4153-ae91-b44a72174b1a" (UID: "3d1e47c1-33b1-4153-ae91-b44a72174b1a"). InnerVolumeSpecName "kube-api-access-5bhq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:22.307580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.307548    4577 generic.go:356] "Generic (PLEG): container finished" podID="3d1e47c1-33b1-4153-ae91-b44a72174b1a" containerID="3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c" exitCode=0
Oct 30 01:09:22.307733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.307596    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"
Oct 30 01:09:22.307733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.307634    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj" event={"ID":"3d1e47c1-33b1-4153-ae91-b44a72174b1a","Type":"ContainerDied","Data":"3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c"}
Oct 30 01:09:22.307733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.307701    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj" event={"ID":"3d1e47c1-33b1-4153-ae91-b44a72174b1a","Type":"ContainerDied","Data":"e5aaa762dfabef745849111278844d0e0f9d430fee7b1289381235dcc299ac37"}
Oct 30 01:09:22.307733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.307731    4577 scope.go:122] "RemoveContainer" containerID="3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c"
Oct 30 01:09:22.309698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.309653    4577 generic.go:356] "Generic (PLEG): container finished" podID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerID="6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099" exitCode=0
Oct 30 01:09:22.309819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.309704    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" event={"ID":"06ef12f9-f0fb-4e0e-b131-7ae244af3855","Type":"ContainerDied","Data":"6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099"}
Oct 30 01:09:22.309819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.309753    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr" event={"ID":"06ef12f9-f0fb-4e0e-b131-7ae244af3855","Type":"ContainerDied","Data":"d38136ab6a1147ee4d5e09631b12e67b8d836605c68e64cd19496c6e1068e484"}
Oct 30 01:09:22.310065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.310050    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"
Oct 30 01:09:22.311511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.311487    4577 generic.go:356] "Generic (PLEG): container finished" podID="4e2b578c-c2df-41db-92a2-406a099c5638" containerID="8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99" exitCode=0
Oct 30 01:09:22.311625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.311517    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"
Oct 30 01:09:22.311625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.311569    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8" event={"ID":"4e2b578c-c2df-41db-92a2-406a099c5638","Type":"ContainerDied","Data":"8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99"}
Oct 30 01:09:22.311625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.311608    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8" event={"ID":"4e2b578c-c2df-41db-92a2-406a099c5638","Type":"ContainerDied","Data":"45f6e8ed7b978600730cfb7e2bf278c7eebdd850df6dcc764f8831e50ce1ebf0"}
Oct 30 01:09:22.315811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.315790    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1e47c1-33b1-4153-ae91-b44a72174b1a-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:22.315915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.315813    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bhq6\" (UniqueName: \"kubernetes.io/projected/3d1e47c1-33b1-4153-ae91-b44a72174b1a-kube-api-access-5bhq6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:22.322111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.322093    4577 scope.go:122] "RemoveContainer" containerID="3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c"
Oct 30 01:09:22.322444 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:22.322421    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c\": container with ID starting with 3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c not found: ID does not exist" containerID="3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c"
Oct 30 01:09:22.322537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.322452    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c"} err="failed to get container status \"3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c\": rpc error: code = NotFound desc = could not find container \"3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c\": container with ID starting with 3a30e7ce20a5177213be224b8bceaca6469ae29f334c7ca20a451e8d2f46d15c not found: ID does not exist"
Oct 30 01:09:22.322537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.322475    4577 scope.go:122] "RemoveContainer" containerID="6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099"
Oct 30 01:09:22.332286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.332268    4577 scope.go:122] "RemoveContainer" containerID="7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"
Oct 30 01:09:22.341993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.341977    4577 scope.go:122] "RemoveContainer" containerID="6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099"
Oct 30 01:09:22.342368 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:22.342341    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099\": container with ID starting with 6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099 not found: ID does not exist" containerID="6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099"
Oct 30 01:09:22.342511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.342375    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099"} err="failed to get container status \"6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099\": rpc error: code = NotFound desc = could not find container \"6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099\": container with ID starting with 6afa3db086a3c6ab8eafcd8ec9c32cd95ea555ef2613390231280a26d8e79099 not found: ID does not exist"
Oct 30 01:09:22.342511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.342398    4577 scope.go:122] "RemoveContainer" containerID="7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"
Oct 30 01:09:22.342711 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:22.342685    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357\": container with ID starting with 7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357 not found: ID does not exist" containerID="7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"
Oct 30 01:09:22.342770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.342730    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357"} err="failed to get container status \"7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357\": rpc error: code = NotFound desc = could not find container \"7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357\": container with ID starting with 7d81cca7f6153cb796cda21fd82786549964c43db7511cf1a5982ceb97f51357 not found: ID does not exist"
Oct 30 01:09:22.342770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.342752    4577 scope.go:122] "RemoveContainer" containerID="8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99"
Oct 30 01:09:22.353916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.353900    4577 scope.go:122] "RemoveContainer" containerID="8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99"
Oct 30 01:09:22.354220 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:22.354186    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99\": container with ID starting with 8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99 not found: ID does not exist" containerID="8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99"
Oct 30 01:09:22.354338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.354217    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99"} err="failed to get container status \"8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99\": rpc error: code = NotFound desc = could not find container \"8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99\": container with ID starting with 8a1c26ee5cf469651f442c07cd85c1f3efc6fa2d108f24c954b571d580f91d99 not found: ID does not exist"
Oct 30 01:09:22.369874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.369836    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"]
Oct 30 01:09:22.376372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.376345    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ad449e-664b8cf45f-fhmsj"]
Oct 30 01:09:22.409928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.409899    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"]
Oct 30 01:09:22.413739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.413715    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ad449e-dcf84c666-t6fh8"]
Oct 30 01:09:22.451525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.451443    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"]
Oct 30 01:09:22.455305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.455281    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ad449e-855d4d9b56-dtwvr"]
Oct 30 01:09:22.574464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:22.574432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:24.063528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:24.063493    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:24.186279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:24.186243    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" path="/var/lib/kubelet/pods/06ef12f9-f0fb-4e0e-b131-7ae244af3855/volumes"
Oct 30 01:09:24.186942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:24.186918    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d1e47c1-33b1-4153-ae91-b44a72174b1a" path="/var/lib/kubelet/pods/3d1e47c1-33b1-4153-ae91-b44a72174b1a/volumes"
Oct 30 01:09:24.187471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:24.187450    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e2b578c-c2df-41db-92a2-406a099c5638" path="/var/lib/kubelet/pods/4e2b578c-c2df-41db-92a2-406a099c5638/volumes"
Oct 30 01:09:26.489221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:26.489193    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:27.020832 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:27.020795    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:09:28.699103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:28.699071    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:28.976773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:28.976637    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:29.095473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:29.095429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:29.283362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:29.283326    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"]
Oct 30 01:09:29.283636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:29.283581    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy" containerID="cri-o://2e6606805ecc4bcc881c4df1b83b022e139b9d4d27a918b7e3e77b9401b66a36" gracePeriod=30
Oct 30 01:09:29.317623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:29.317593    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:30.458871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:30.457833    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:30.831334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:30.831302    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:31.230621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.230588    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"]
Oct 30 01:09:31.231616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231583    4577 cpu_manager.go:414] "containerMap: removing container" podUID="3d1e47c1-33b1-4153-ae91-b44a72174b1a" containerName="payload-pre-processing"
Oct 30 01:09:31.231616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231608    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d1e47c1-33b1-4153-ae91-b44a72174b1a" containerName="payload-pre-processing"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231628    4577 cpu_manager.go:414] "containerMap: removing container" podUID="4e2b578c-c2df-41db-92a2-406a099c5638" containerName="payload-processing"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231636    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e2b578c-c2df-41db-92a2-406a099c5638" containerName="payload-processing"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231670    4577 cpu_manager.go:414] "containerMap: removing container" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231677    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231703    4577 cpu_manager.go:414] "containerMap: removing container" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231710    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231723    4577 cpu_manager.go:414] "containerMap: removing container" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231730    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231742    4577 cpu_manager.go:414] "containerMap: removing container" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api"
Oct 30 01:09:31.231834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231750    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api"
Oct 30 01:09:31.232465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231971    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d1e47c1-33b1-4153-ae91-b44a72174b1a" containerName="payload-pre-processing"
Oct 30 01:09:31.232465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.231986    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e2b578c-c2df-41db-92a2-406a099c5638" containerName="payload-processing"
Oct 30 01:09:31.232465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.232006    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy"
Oct 30 01:09:31.232465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.232017    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="47dcdb1c-2440-4ffc-aecd-8f7dae131d12" containerName="istio-proxy"
Oct 30 01:09:31.232465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.232026    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api"
Oct 30 01:09:31.232465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.232035    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="06ef12f9-f0fb-4e0e-b131-7ae244af3855" containerName="maas-api"
Oct 30 01:09:31.237380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.237361    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.295351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.295313    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:31.296398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.296377    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"]
Oct 30 01:09:31.297118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.297101    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-ce5996-openshift-default-dockercfg-vp2nf\""
Oct 30 01:09:31.391018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.390984    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391028    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391099    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391258    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391371    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391434    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391503    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391525    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbbj2\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.391641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.391580    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.420540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.420507    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:31.495415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.495314    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.496402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.496377    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbbj2\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.507310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.507276    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.508135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.508099    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.508916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.508893    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.509093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.509076    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.517483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.517455    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.518144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.518118    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.518785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.518762    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.521534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.517945    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.529116 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.507953    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.529307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.518577    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.529412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.517234    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.529509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.519668    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.543340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.543308    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbbj2\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.551870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.549376    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.584347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.573866    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.584347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.584138    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy\") pod \"e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") " pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:31.609460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:31.609138    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:32.074427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.074400    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"]
Oct 30 01:09:32.075733 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:09:32.075706    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a46170_e759_428e_aa74_dbf88c1fdd26.slice/crio-f0e1e74fa7f0b09d7c613a198985705b2ac0bac20c7bfa4480aa0b39dabf31bd WatchSource:0}: Error finding container f0e1e74fa7f0b09d7c613a198985705b2ac0bac20c7bfa4480aa0b39dabf31bd: Status 404 returned error can't find the container with id f0e1e74fa7f0b09d7c613a198985705b2ac0bac20c7bfa4480aa0b39dabf31bd
Oct 30 01:09:32.078799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.078765    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:32.078917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.078840    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:32.078917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.078893    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:32.408065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.407973    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" event={"ID":"c1a46170-e759-428e-aa74-dbf88c1fdd26","Type":"ContainerStarted","Data":"13321903ceee490d5a1bc4039a0bd595312560850b81d3c17c162d0f582b2052"}
Oct 30 01:09:32.408065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.408050    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" event={"ID":"c1a46170-e759-428e-aa74-dbf88c1fdd26","Type":"ContainerStarted","Data":"f0e1e74fa7f0b09d7c613a198985705b2ac0bac20c7bfa4480aa0b39dabf31bd"}
Oct 30 01:09:32.487833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.487416    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:32.488306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.488260    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"]
Oct 30 01:09:32.500408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.500338    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podStartSLOduration=1.500312712 podStartE2EDuration="1.500312712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:32.486138089 +0000 UTC m=+1680.515448320" watchObservedRunningTime="2026-10-30 01:09:32.500312712 +0000 UTC m=+1680.529622944"
Oct 30 01:09:32.604689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.604650    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:32.616290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.613458    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:32.735221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:32.735102    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:33.619908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:33.619878    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:34.426905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.426836    4577 generic.go:356] "Generic (PLEG): container finished" podID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerID="2e6606805ecc4bcc881c4df1b83b022e139b9d4d27a918b7e3e77b9401b66a36" exitCode=0
Oct 30 01:09:34.426905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.426895    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" event={"ID":"72aee840-e2dd-4599-a7db-ab3ee0b2098a","Type":"ContainerDied","Data":"2e6606805ecc4bcc881c4df1b83b022e139b9d4d27a918b7e3e77b9401b66a36"}
Oct 30 01:09:34.427258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.426967    4577 scope.go:122] "RemoveContainer" containerID="8a69303d37b73ccd7c4036a87512a4296d1a89dad626ed369bcb57b815aa6753"
Oct 30 01:09:34.427793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.427607    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:34.429304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.428678    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:09:34.482388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.482322    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:34.788872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.787204    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:09:34.827741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.827704    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.827958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.827867    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.827958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.827907    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.827959    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828030    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:34.828200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828097    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828177    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828238    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:34.828301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828272    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828374    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828451    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo\") pod \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\" (UID: \"72aee840-e2dd-4599-a7db-ab3ee0b2098a\") "
Oct 30 01:09:34.828538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828517    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:34.828595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828534    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:34.828762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.828734    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:34.829239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.829099    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.829239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.829124    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.829239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.829138    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.829239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.829152    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.829239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.829166    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.831103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.830874    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:09:34.831103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.830979    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:34.831103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.831003    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:34.831476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.831459    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n" pod "72aee840-e2dd-4599-a7db-ab3ee0b2098a" (UID: "72aee840-e2dd-4599-a7db-ab3ee0b2098a"). InnerVolumeSpecName "kube-api-access-tls4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:34.930466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.930360    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tls4n\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-kube-api-access-tls4n\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.930466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.930413    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.930466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.930423    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.930466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.930430    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72aee840-e2dd-4599-a7db-ab3ee0b2098a-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:34.969868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:34.969453    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:35.102618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:35.102583    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:35.436091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:35.436059    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"
Oct 30 01:09:35.436261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:35.436105    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb" event={"ID":"72aee840-e2dd-4599-a7db-ab3ee0b2098a","Type":"ContainerDied","Data":"8aecaeebd1d9a9e4ffbdc0e4efb11161a6cbfb77589f0d42aed2f252596ffdde"}
Oct 30 01:09:35.436261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:35.436169    4577 scope.go:122] "RemoveContainer" containerID="2e6606805ecc4bcc881c4df1b83b022e139b9d4d27a918b7e3e77b9401b66a36"
Oct 30 01:09:35.505459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:35.505423    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"]
Oct 30 01:09:35.521034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:35.521008    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-ad449e-openshift-default-d5dc56bdc-c8qhb"]
Oct 30 01:09:36.113113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.113084    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" path="/var/lib/kubelet/pods/72aee840-e2dd-4599-a7db-ab3ee0b2098a/volumes"
Oct 30 01:09:36.625765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.625715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:36.743292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.743258    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"]
Oct 30 01:09:36.744096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.744078    4577 cpu_manager.go:414] "containerMap: removing container" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy"
Oct 30 01:09:36.744165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.744097    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy"
Oct 30 01:09:36.744165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.744106    4577 cpu_manager.go:414] "containerMap: removing container" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy"
Oct 30 01:09:36.744165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.744112    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy"
Oct 30 01:09:36.744264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.744214    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy"
Oct 30 01:09:36.744264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.744221    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="72aee840-e2dd-4599-a7db-ab3ee0b2098a" containerName="istio-proxy"
Oct 30 01:09:36.754724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.754693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:36.758653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.758629    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:36.772151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.769871    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"]
Oct 30 01:09:36.775704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.775676    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-ce5996\""
Oct 30 01:09:36.835033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.835002    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"]
Oct 30 01:09:36.839373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.839346    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:36.859123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.857567    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-ce5996-dockercfg-dq9jn\""
Oct 30 01:09:36.859123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.858527    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:36.859123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.858690    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:36.859123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.858781    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5rc\" (UniqueName: \"kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc\") pod \"payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:36.859123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.858941    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5k6x\" (UniqueName: \"kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x\") pod \"payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:36.958922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:36.943547    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"]
Oct 30 01:09:37.015653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.015610    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5rc\" (UniqueName: \"kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc\") pod \"payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:37.015869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.015755    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5k6x\" (UniqueName: \"kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x\") pod \"payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:37.015869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.015825    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:37.016086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.016051    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:37.063093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.055480    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:37.063093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.055797    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:37.156539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.156498    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5k6x\" (UniqueName: \"kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x\") pod \"payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:37.157491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.157231    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5rc\" (UniqueName: \"kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc\") pod \"payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:37.161044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.161019    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:37.331047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.331009    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:37.393184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.392895    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:37.488018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.487922    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"]
Oct 30 01:09:37.490256 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:09:37.490221    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602fe27f_22fd_4b57_ab73_dbbad8e3b6f0.slice/crio-c3d3efb241af43614a9a187b805d427b458ea2d33d08f6b6bd118f9eaa3f1608 WatchSource:0}: Error finding container c3d3efb241af43614a9a187b805d427b458ea2d33d08f6b6bd118f9eaa3f1608: Status 404 returned error can't find the container with id c3d3efb241af43614a9a187b805d427b458ea2d33d08f6b6bd118f9eaa3f1608
Oct 30 01:09:37.493506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.493458    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x" podUID="b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" containerName="payload-pre-processing" containerID="cri-o://607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796" gracePeriod=25
Oct 30 01:09:37.778539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.778502    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"]
Oct 30 01:09:37.793883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.793810    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:09:37.857170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.855771    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw\" (UniqueName: \"kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw\") pod \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") "
Oct 30 01:09:37.857170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.855929    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume\") pod \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\" (UID: \"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505\") "
Oct 30 01:09:37.857585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.857547    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume" pod "b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" (UID: "b76db7b4-27aa-4be2-9ccc-6d07b3ab1505"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:37.863384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.863348    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw" pod "b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" (UID: "b76db7b4-27aa-4be2-9ccc-6d07b3ab1505"). InnerVolumeSpecName "kube-api-access-jxjxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:37.958702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.958655    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:37.958702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:37.958694    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxjxw\" (UniqueName: \"kubernetes.io/projected/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505-kube-api-access-jxjxw\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:38.467922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.467723    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" event={"ID":"9ccd6567-e409-4ae4-9b4a-be9d8debf002","Type":"ContainerStarted","Data":"e7e72ac6fad90199fc94b88c9180140f0782ecee80028d00f8c7583f7babe5c8"}
Oct 30 01:09:38.467922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.467790    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" event={"ID":"9ccd6567-e409-4ae4-9b4a-be9d8debf002","Type":"ContainerStarted","Data":"cc6f6a60ebcd31adc7a4321465ab0500c1f4d0af58a161669983b2033383f877"}
Oct 30 01:09:38.467922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.467819    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:38.470311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.470263    4577 generic.go:356] "Generic (PLEG): container finished" podID="b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" containerID="607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796" exitCode=0
Oct 30 01:09:38.470311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.470306    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"
Oct 30 01:09:38.470524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.470300    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x" event={"ID":"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505","Type":"ContainerDied","Data":"607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796"}
Oct 30 01:09:38.470524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.470439    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x" event={"ID":"b76db7b4-27aa-4be2-9ccc-6d07b3ab1505","Type":"ContainerDied","Data":"42c6fa7b44e7797bea895dc43cb075ac00a8e7461faac53c3110b04cb55f47d2"}
Oct 30 01:09:38.470524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.470475    4577 scope.go:122] "RemoveContainer" containerID="607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796"
Oct 30 01:09:38.472915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.472837    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" event={"ID":"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0","Type":"ContainerStarted","Data":"db3ebf0a5b4239725fbb9f01abf39e472dea9503a717eb68337f49f8066d0573"}
Oct 30 01:09:38.473145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.472943    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" event={"ID":"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0","Type":"ContainerStarted","Data":"c3d3efb241af43614a9a187b805d427b458ea2d33d08f6b6bd118f9eaa3f1608"}
Oct 30 01:09:38.473145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.473048    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:38.485435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.485410    4577 scope.go:122] "RemoveContainer" containerID="607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796"
Oct 30 01:09:38.485797 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:38.485771    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796\": container with ID starting with 607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796 not found: ID does not exist" containerID="607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796"
Oct 30 01:09:38.485944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.485805    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796"} err="failed to get container status \"607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796\": rpc error: code = NotFound desc = could not find container \"607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796\": container with ID starting with 607268df0200e4fcfb230659ee43173880f8dedc80d957687c285cf7bb7df796 not found: ID does not exist"
Oct 30 01:09:38.514763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.514698    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" podStartSLOduration=2.514683452 podStartE2EDuration="2.514683452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:38.512099673 +0000 UTC m=+1686.541409904" watchObservedRunningTime="2026-10-30 01:09:38.514683452 +0000 UTC m=+1686.543993719"
Oct 30 01:09:38.558398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.558341    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" podStartSLOduration=2.558325676 podStartE2EDuration="2.558325676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:38.556530285 +0000 UTC m=+1686.585840534" watchObservedRunningTime="2026-10-30 01:09:38.558325676 +0000 UTC m=+1686.587635907"
Oct 30 01:09:38.600524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.600482    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"]
Oct 30 01:09:38.608551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:38.608519    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5bf6b9dbff-mch2x"]
Oct 30 01:09:39.120690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:39.120646    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:40.113717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:40.113678    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" path="/var/lib/kubelet/pods/b76db7b4-27aa-4be2-9ccc-6d07b3ab1505/volumes"
Oct 30 01:09:40.930583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:40.930556    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:41.500400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:41.500363    4577 generic.go:356] "Generic (PLEG): container finished" podID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerID="89e78379fa5969b5f067c26c1b09f7cb06513e5b7cc1b43ac9faec57aeb93acd" exitCode=0
Oct 30 01:09:41.500795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:41.500415    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" event={"ID":"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6","Type":"ContainerDied","Data":"89e78379fa5969b5f067c26c1b09f7cb06513e5b7cc1b43ac9faec57aeb93acd"}
Oct 30 01:09:41.500863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:41.500822    4577 scope.go:122] "RemoveContainer" containerID="89e78379fa5969b5f067c26c1b09f7cb06513e5b7cc1b43ac9faec57aeb93acd"
Oct 30 01:09:41.502397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:41.502371    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:41.502462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:41.502436    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:41.502498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:41.502469    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:42.062681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.062642    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:42.510130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.510094    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" event={"ID":"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6","Type":"ContainerStarted","Data":"19aefa00ad9669bde87b9ffc79f7b7641e0bca2f0bfd96e9e625ea589b97d72c"}
Oct 30 01:09:42.857929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.855017    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"]
Oct 30 01:09:42.857929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.856175    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" containerName="payload-pre-processing"
Oct 30 01:09:42.857929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.856196    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" containerName="payload-pre-processing"
Oct 30 01:09:42.857929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.856363    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b76db7b4-27aa-4be2-9ccc-6d07b3ab1505" containerName="payload-pre-processing"
Oct 30 01:09:42.867870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.864200    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:42.867870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.867630    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"]
Oct 30 01:09:42.891873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.888669    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-4ec143-serving-cert\""
Oct 30 01:09:42.996708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:42.996658    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:43.174923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.147462    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.208061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.207675    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.208942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.208916    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chdqs\" (UniqueName: \"kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.209231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.176191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:43.209590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.209254    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.310937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.310887    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.310937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.310943    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.311731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.311282    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.311731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.311366    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chdqs\" (UniqueName: \"kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.312187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.312161    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.314335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.314310    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.314555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.314524    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.327943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.327902    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:09:43.328183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.328163    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 01:09:43.328271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.328219    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 01:09:43.336239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.336206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:43.338702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.338676    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chdqs\" (UniqueName: \"kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs\") pod \"maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.606145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.606098    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:43.981560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:43.981515    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"]
Oct 30 01:09:44.327567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.327532    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 01:09:44.327710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.327583    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 01:09:44.528616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.528547    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96" event={"ID":"1c5a124b-126f-425a-bbbe-92433b0981e3","Type":"ContainerStarted","Data":"affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb"}
Oct 30 01:09:44.528786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.528643    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96" event={"ID":"1c5a124b-126f-425a-bbbe-92433b0981e3","Type":"ContainerStarted","Data":"f17d0b6a4bd2c42a2583a9c4d05bde5030b0d6cefab1c01db404804408832d9d"}
Oct 30 01:09:44.528786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.528702    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:44.584768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.584668    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96" podStartSLOduration=2.16390884 podStartE2EDuration="2.584643017s" totalImagesPullingTime="420.734177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:44.583548841 +0000 UTC m=+1692.612859072" watchObservedRunningTime="2026-10-30 01:09:44.584643017 +0000 UTC m=+1692.613953277"
Oct 30 01:09:44.812694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:44.812657    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:45.327359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:45.327325    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 01:09:45.327569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:45.327392    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 01:09:46.327679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:46.327644    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 01:09:46.328127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:46.327692    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 01:09:46.873618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:46.873584    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:47.327940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:47.327906    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 01:09:47.328359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:47.327970    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 01:09:48.331817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:48.331777    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:09:48.332337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:48.332267    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:09:48.333060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:48.333044    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:09:48.732910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:48.732877    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:49.486264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:49.486235    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:09:49.486747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:49.486316    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:09:50.461064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:50.461026    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:50.558290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:50.558261    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:09:52.482907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.482315    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"]
Oct 30 01:09:52.482907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.482579    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45" podUID="1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" containerName="maas-api" containerID="cri-o://c8d2b68ae72d061403fc903dc808c54f5fd73593784aa0fba67b7391b4f4b518" gracePeriod=30
Oct 30 01:09:52.547770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.547736    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:52.649097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.649062    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"]
Oct 30 01:09:52.649422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.649353    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing" containerID="cri-o://137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471" gracePeriod=30
Oct 30 01:09:52.675322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.675210    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"]
Oct 30 01:09:52.675561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.675407    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9" podUID="6c4c8502-4e79-46f5-9f41-84f65c8c3350" containerName="payload-pre-processing" containerID="cri-o://ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c" gracePeriod=30
Oct 30 01:09:52.681344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.681163    4577 generic.go:356] "Generic (PLEG): container finished" podID="1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" containerID="c8d2b68ae72d061403fc903dc808c54f5fd73593784aa0fba67b7391b4f4b518" exitCode=0
Oct 30 01:09:52.681344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.681264    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45" event={"ID":"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1","Type":"ContainerDied","Data":"c8d2b68ae72d061403fc903dc808c54f5fd73593784aa0fba67b7391b4f4b518"}
Oct 30 01:09:52.738395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.738313    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:52.855049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.852150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:52.895306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.895274    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:09:52.995097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.989924    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn\" (UniqueName: \"kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn\") pod \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") "
Oct 30 01:09:52.995097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.990066    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs\") pod \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") "
Oct 30 01:09:52.995097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.990210    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca\") pod \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") "
Oct 30 01:09:52.995097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:52.990288    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls\") pod \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\" (UID: \"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1\") "
Oct 30 01:09:53.033986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.017267    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls" pod "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" (UID: "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:53.033986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.017647    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca" pod "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" (UID: "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:53.033986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.018021    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn" pod "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" (UID: "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1"). InnerVolumeSpecName "kube-api-access-pctbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:53.043024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.038360    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs" pod "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" (UID: "1dbf5831-1044-4ef5-92c4-940cb0d3cdd1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:53.102869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.096107    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.102869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.096142    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pctbn\" (UniqueName: \"kubernetes.io/projected/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-kube-api-access-pctbn\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.102869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.096160    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.102869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.096175    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.124837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.117411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:53.233891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.233549    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:53.242885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.242013    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:09:53.264320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.264282    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:09:53.302423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.302381    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl\" (UniqueName: \"kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl\") pod \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") "
Oct 30 01:09:53.303320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.303295    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume\") pod \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\" (UID: \"d7c25040-db18-4255-9d25-7fe1a5aa0b5b\") "
Oct 30 01:09:53.304089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.303963    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume" pod "d7c25040-db18-4255-9d25-7fe1a5aa0b5b" (UID: "d7c25040-db18-4255-9d25-7fe1a5aa0b5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:53.304687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.304032    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz\" (UniqueName: \"kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz\") pod \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") "
Oct 30 01:09:53.305524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.304997    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume\") pod \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\" (UID: \"6c4c8502-4e79-46f5-9f41-84f65c8c3350\") "
Oct 30 01:09:53.305735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.305503    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume" pod "6c4c8502-4e79-46f5-9f41-84f65c8c3350" (UID: "6c4c8502-4e79-46f5-9f41-84f65c8c3350"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:53.306445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.306422    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.306585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.306569    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4c8502-4e79-46f5-9f41-84f65c8c3350-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.326153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.326074    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl" pod "d7c25040-db18-4255-9d25-7fe1a5aa0b5b" (UID: "d7c25040-db18-4255-9d25-7fe1a5aa0b5b"). InnerVolumeSpecName "kube-api-access-qhknl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:53.340312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.340224    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz" pod "6c4c8502-4e79-46f5-9f41-84f65c8c3350" (UID: "6c4c8502-4e79-46f5-9f41-84f65c8c3350"). InnerVolumeSpecName "kube-api-access-6tpnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:53.366883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.366082    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:53.408601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.408526    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhknl\" (UniqueName: \"kubernetes.io/projected/d7c25040-db18-4255-9d25-7fe1a5aa0b5b-kube-api-access-qhknl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.408601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.408563    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tpnz\" (UniqueName: \"kubernetes.io/projected/6c4c8502-4e79-46f5-9f41-84f65c8c3350-kube-api-access-6tpnz\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.700277    4577 generic.go:356] "Generic (PLEG): container finished" podID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerID="137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471" exitCode=0
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.700577    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.701234    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" event={"ID":"d7c25040-db18-4255-9d25-7fe1a5aa0b5b","Type":"ContainerDied","Data":"137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471"}
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.701287    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z" event={"ID":"d7c25040-db18-4255-9d25-7fe1a5aa0b5b","Type":"ContainerDied","Data":"74ba3f65baf587ec1d7b1246f5cad2daa08bb0f9ce27a37e62e860eee50cdfc9"}
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.701318    4577 scope.go:122] "RemoveContainer" containerID="137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471"
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.704015    4577 generic.go:356] "Generic (PLEG): container finished" podID="6c4c8502-4e79-46f5-9f41-84f65c8c3350" containerID="ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c" exitCode=0
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.704143    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9" event={"ID":"6c4c8502-4e79-46f5-9f41-84f65c8c3350","Type":"ContainerDied","Data":"ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c"}
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.704188    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9" event={"ID":"6c4c8502-4e79-46f5-9f41-84f65c8c3350","Type":"ContainerDied","Data":"1b4c0fa725b586cdb470748997be5165ec405a710226b85ca30e59190c74d094"}
Oct 30 01:09:53.705870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.704273    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"
Oct 30 01:09:53.710863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.710303    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45" event={"ID":"1dbf5831-1044-4ef5-92c4-940cb0d3cdd1","Type":"ContainerDied","Data":"93e082fc1f0d854f9bbf0d7f0bd20c1dd2567759ef62c63e52558cf3d8cf4b53"}
Oct 30 01:09:53.710863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.710392    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"
Oct 30 01:09:53.720967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.720942    4577 scope.go:122] "RemoveContainer" containerID="ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"
Oct 30 01:09:53.733042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.732968    4577 scope.go:122] "RemoveContainer" containerID="137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471"
Oct 30 01:09:53.733592 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:53.733280    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471\": container with ID starting with 137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471 not found: ID does not exist" containerID="137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471"
Oct 30 01:09:53.733592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.733327    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471"} err="failed to get container status \"137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471\": rpc error: code = NotFound desc = could not find container \"137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471\": container with ID starting with 137dd5a631a88aea7c61f4f802ebc67aaabac192c710def2753cbe97c632e471 not found: ID does not exist"
Oct 30 01:09:53.733592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.733350    4577 scope.go:122] "RemoveContainer" containerID="ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"
Oct 30 01:09:53.733813 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:53.733648    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79\": container with ID starting with ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79 not found: ID does not exist" containerID="ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"
Oct 30 01:09:53.733813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.733673    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79"} err="failed to get container status \"ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79\": rpc error: code = NotFound desc = could not find container \"ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79\": container with ID starting with ab9ae669e28c3c61e4507447ce734fc77236532ed8a24ef9eb2c33786ba57e79 not found: ID does not exist"
Oct 30 01:09:53.733813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.733688    4577 scope.go:122] "RemoveContainer" containerID="ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c"
Oct 30 01:09:53.752537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.752515    4577 scope.go:122] "RemoveContainer" containerID="ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c"
Oct 30 01:09:53.752907 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:53.752880    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c\": container with ID starting with ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c not found: ID does not exist" containerID="ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c"
Oct 30 01:09:53.753027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.752917    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c"} err="failed to get container status \"ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c\": rpc error: code = NotFound desc = could not find container \"ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c\": container with ID starting with ca144894b7600a70af968ba6bc5f96f6b4de11edf42063458204e714531e3a7c not found: ID does not exist"
Oct 30 01:09:53.753027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.752941    4577 scope.go:122] "RemoveContainer" containerID="c8d2b68ae72d061403fc903dc808c54f5fd73593784aa0fba67b7391b4f4b518"
Oct 30 01:09:53.806925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.806855    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"]
Oct 30 01:09:53.838050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.838022    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a8ab86-757f6c45b7-h2cd9"]
Oct 30 01:09:53.873547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.873520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:53.891235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.891205    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"]
Oct 30 01:09:53.906715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.901381    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a8ab86-6bdc5ddb98-7x65z"]
Oct 30 01:09:53.958653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.958622    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"]
Oct 30 01:09:53.962197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:53.962177    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a8ab86-7cc68f7d9b-m8s45"]
Oct 30 01:09:54.113718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:54.113644    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" path="/var/lib/kubelet/pods/1dbf5831-1044-4ef5-92c4-940cb0d3cdd1/volumes"
Oct 30 01:09:54.114135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:54.114121    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c4c8502-4e79-46f5-9f41-84f65c8c3350" path="/var/lib/kubelet/pods/6c4c8502-4e79-46f5-9f41-84f65c8c3350/volumes"
Oct 30 01:09:54.114484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:54.114472    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" path="/var/lib/kubelet/pods/d7c25040-db18-4255-9d25-7fe1a5aa0b5b/volumes"
Oct 30 01:09:54.998322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:54.998291    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:55.226787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.222323    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.566663    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"]
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567671    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567689    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567711    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6c4c8502-4e79-46f5-9f41-84f65c8c3350" containerName="payload-pre-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567719    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c4c8502-4e79-46f5-9f41-84f65c8c3350" containerName="payload-pre-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567750    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567756    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567775    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" containerName="maas-api"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567781    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" containerName="maas-api"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.567989    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.568005    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c4c8502-4e79-46f5-9f41-84f65c8c3350" containerName="payload-pre-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.568019    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c25040-db18-4255-9d25-7fe1a5aa0b5b" containerName="payload-processing"
Oct 30 01:09:55.571898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.568030    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1dbf5831-1044-4ef5-92c4-940cb0d3cdd1" containerName="maas-api"
Oct 30 01:09:55.573974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.573917    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.607801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.603654    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.607801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.603974    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vghrf\" (UniqueName: \"kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf\") pod \"payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.607801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.605188    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d3c1fa\""
Oct 30 01:09:55.607801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.606625    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"]
Oct 30 01:09:55.679599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.679272    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:55.689005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.688106    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d3c1fa-dockercfg-zz6ng\""
Oct 30 01:09:55.719369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.706462    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.719369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.706747    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vghrf\" (UniqueName: \"kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf\") pod \"payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.719369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.708199    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.739157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.735539    4577 generic.go:356] "Generic (PLEG): container finished" podID="0dd9de74-415f-46de-849d-d846276afdb7" containerID="838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772" exitCode=0
Oct 30 01:09:55.739157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.735801    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" event={"ID":"0dd9de74-415f-46de-849d-d846276afdb7","Type":"ContainerDied","Data":"838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"}
Oct 30 01:09:55.739157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.736395    4577 scope.go:122] "RemoveContainer" containerID="838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"
Oct 30 01:09:55.739921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.739392    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:55.739921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.739477    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:55.739921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.739530    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:09:55.744406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.743923    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vghrf\" (UniqueName: \"kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf\") pod \"payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.788684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.788640    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"]
Oct 30 01:09:55.925678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.925392    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:55.951643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.951520    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:55.954546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.953737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:55.979363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:55.979260    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"]
Oct 30 01:09:56.049695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.046435    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"]
Oct 30 01:09:56.049695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.046678    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc" podUID="311b2aa4-f32a-4482-9c7d-207f52e599f2" containerName="maas-api" containerID="cri-o://f2cf9a994ad4d03c14686e13a4779e9db532f28cc6974274886b1652f740940c" gracePeriod=30
Oct 30 01:09:56.056351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.056100    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhqvq\" (UniqueName: \"kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.056351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.056202    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.135433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.135023    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:56.166739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.165641    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhqvq\" (UniqueName: \"kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.166739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.165721    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.166739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.166447    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.219897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.218196    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhqvq\" (UniqueName: \"kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq\") pod \"payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.472949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.368429    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:56.662867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.563017    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"]
Oct 30 01:09:56.836084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.836050    4577 generic.go:356] "Generic (PLEG): container finished" podID="311b2aa4-f32a-4482-9c7d-207f52e599f2" containerID="f2cf9a994ad4d03c14686e13a4779e9db532f28cc6974274886b1652f740940c" exitCode=0
Oct 30 01:09:56.836223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.836161    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc" event={"ID":"311b2aa4-f32a-4482-9c7d-207f52e599f2","Type":"ContainerDied","Data":"f2cf9a994ad4d03c14686e13a4779e9db532f28cc6974274886b1652f740940c"}
Oct 30 01:09:56.839506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.839426    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" event={"ID":"0dd9de74-415f-46de-849d-d846276afdb7","Type":"ContainerStarted","Data":"bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8"}
Oct 30 01:09:56.841995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.841936    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm" event={"ID":"25441484-890b-4e6c-94e0-b4dd99376b8c","Type":"ContainerStarted","Data":"d5584962f1132b1a4ef8ee16334b380889ef8fce09da02470d3d9fffd2c142cb"}
Oct 30 01:09:56.933479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:56.933400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:57.049341 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:09:57.049246    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:09:57.200239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.200212    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:09:57.287554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.287519    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs\") pod \"311b2aa4-f32a-4482-9c7d-207f52e599f2\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") "
Oct 30 01:09:57.287756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.287676    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca\") pod \"311b2aa4-f32a-4482-9c7d-207f52e599f2\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") "
Oct 30 01:09:57.288156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.287835    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s\" (UniqueName: \"kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s\") pod \"311b2aa4-f32a-4482-9c7d-207f52e599f2\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") "
Oct 30 01:09:57.288156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.287960    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls\") pod \"311b2aa4-f32a-4482-9c7d-207f52e599f2\" (UID: \"311b2aa4-f32a-4482-9c7d-207f52e599f2\") "
Oct 30 01:09:57.288156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.288058    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca" pod "311b2aa4-f32a-4482-9c7d-207f52e599f2" (UID: "311b2aa4-f32a-4482-9c7d-207f52e599f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:57.288541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.288510    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/311b2aa4-f32a-4482-9c7d-207f52e599f2-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:57.290769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.290728    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s" pod "311b2aa4-f32a-4482-9c7d-207f52e599f2" (UID: "311b2aa4-f32a-4482-9c7d-207f52e599f2"). InnerVolumeSpecName "kube-api-access-jqh6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:57.291003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.290973    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls" pod "311b2aa4-f32a-4482-9c7d-207f52e599f2" (UID: "311b2aa4-f32a-4482-9c7d-207f52e599f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:57.291503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.291455    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs" pod "311b2aa4-f32a-4482-9c7d-207f52e599f2" (UID: "311b2aa4-f32a-4482-9c7d-207f52e599f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:09:57.350678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.350640    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"]
Oct 30 01:09:57.352059 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:09:57.352030    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06b60da_d546_4ce9_ac26_7b38882a18d5.slice/crio-90e898f4e88b3d4fb5fe4b4c5374546e48d17197a2f4da1629f36a253a1e831f WatchSource:0}: Error finding container 90e898f4e88b3d4fb5fe4b4c5374546e48d17197a2f4da1629f36a253a1e831f: Status 404 returned error can't find the container with id 90e898f4e88b3d4fb5fe4b4c5374546e48d17197a2f4da1629f36a253a1e831f
Oct 30 01:09:57.389681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.389648    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:57.389681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.389680    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqh6s\" (UniqueName: \"kubernetes.io/projected/311b2aa4-f32a-4482-9c7d-207f52e599f2-kube-api-access-jqh6s\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:57.389977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.389696    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/311b2aa4-f32a-4482-9c7d-207f52e599f2-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:09:57.565538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.565454    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:09:57.567124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.567092    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 01:09:57.567262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.567151    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 01:09:57.877969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.877453    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654" event={"ID":"d06b60da-d546-4ce9-ac26-7b38882a18d5","Type":"ContainerStarted","Data":"e941a2c409b5927c58789cd26180991ec1a36f5257badb70819a2be1f3b71c6e"}
Oct 30 01:09:57.877969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.877523    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654" event={"ID":"d06b60da-d546-4ce9-ac26-7b38882a18d5","Type":"ContainerStarted","Data":"90e898f4e88b3d4fb5fe4b4c5374546e48d17197a2f4da1629f36a253a1e831f"}
Oct 30 01:09:57.883192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.878697    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:09:57.884702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.884676    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm" event={"ID":"25441484-890b-4e6c-94e0-b4dd99376b8c","Type":"ContainerStarted","Data":"ea5d7ce7fa74a8ee53b6777374250ce76e132009022b89f28d9dcbc9ea5c99b4"}
Oct 30 01:09:57.885376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.885358    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:09:57.887984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.887963    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"
Oct 30 01:09:57.888114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.888004    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc" event={"ID":"311b2aa4-f32a-4482-9c7d-207f52e599f2","Type":"ContainerDied","Data":"6eb010b47ee222a7f6ce4a1029ef9fed15a87c3405b863afd245f2f8a6c3ec87"}
Oct 30 01:09:57.888114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.888066    4577 scope.go:122] "RemoveContainer" containerID="f2cf9a994ad4d03c14686e13a4779e9db532f28cc6974274886b1652f740940c"
Oct 30 01:09:57.944415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.944359    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654" podStartSLOduration=2.944344221 podStartE2EDuration="2.944344221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:57.939695413 +0000 UTC m=+1705.969005652" watchObservedRunningTime="2026-10-30 01:09:57.944344221 +0000 UTC m=+1705.973654458"
Oct 30 01:09:57.990233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.990198    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"]
Oct 30 01:09:57.994061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:57.994034    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b43bff-c9966b869-xgmfc"]
Oct 30 01:09:58.037122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.037064    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm" podStartSLOduration=3.037046089 podStartE2EDuration="3.037046089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:58.03511867 +0000 UTC m=+1706.064428903" watchObservedRunningTime="2026-10-30 01:09:58.037046089 +0000 UTC m=+1706.066356324"
Oct 30 01:09:58.112409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.112382    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="311b2aa4-f32a-4482-9c7d-207f52e599f2" path="/var/lib/kubelet/pods/311b2aa4-f32a-4482-9c7d-207f52e599f2/volumes"
Oct 30 01:09:58.565869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.565291    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 01:09:58.565869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.565371    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 01:09:58.888336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.888245    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"]
Oct 30 01:09:58.889330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.889308    4577 cpu_manager.go:414] "containerMap: removing container" podUID="311b2aa4-f32a-4482-9c7d-207f52e599f2" containerName="maas-api"
Oct 30 01:09:58.889330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.889327    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="311b2aa4-f32a-4482-9c7d-207f52e599f2" containerName="maas-api"
Oct 30 01:09:58.889519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.889503    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="311b2aa4-f32a-4482-9c7d-207f52e599f2" containerName="maas-api"
Oct 30 01:09:58.895329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.895308    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:58.920884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.915465    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"]
Oct 30 01:09:58.942899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.942147    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-ce5996-serving-cert\""
Oct 30 01:09:58.989648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:58.989609    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:59.024459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.024429    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.024629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.024491    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.024629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.024541    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.024629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.024624    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxmpq\" (UniqueName: \"kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.104332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.104296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:59.130670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.129953    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.130670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.130025    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.130670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.130092    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.130670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.130221    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxmpq\" (UniqueName: \"kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.176919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.149896    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.177823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.165752    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.178020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.177734    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.206891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.206452    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxmpq\" (UniqueName: \"kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq\") pod \"maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.223867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.217117    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:09:59.240080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.240032    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:09:59.568879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.565264    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 01:09:59.568879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.565329    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 01:09:59.621410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.621378    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"]
Oct 30 01:09:59.621663 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:09:59.621633    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957197fd_0c92_4c61_828a_3eb3e81c6b9d.slice/crio-1527ec87d929f7b8f87dea2d3de253a5f29635287fc9597bbceffa6e72d67c23 WatchSource:0}: Error finding container 1527ec87d929f7b8f87dea2d3de253a5f29635287fc9597bbceffa6e72d67c23: Status 404 returned error can't find the container with id 1527ec87d929f7b8f87dea2d3de253a5f29635287fc9597bbceffa6e72d67c23
Oct 30 01:09:59.938077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:09:59.937049    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq" event={"ID":"957197fd-0c92-4c61-828a-3eb3e81c6b9d","Type":"ContainerStarted","Data":"1527ec87d929f7b8f87dea2d3de253a5f29635287fc9597bbceffa6e72d67c23"}
Oct 30 01:10:00.564873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:00.562083    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:10:00.568870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:00.566224    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 01:10:00.568870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:00.566280    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 01:10:00.914965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:00.914821    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:01.026372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.026335    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"]
Oct 30 01:10:01.028037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.028005    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy" containerID="cri-o://39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e" gracePeriod=30
Oct 30 01:10:01.030762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.030726    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq" event={"ID":"957197fd-0c92-4c61-828a-3eb3e81c6b9d","Type":"ContainerStarted","Data":"5d0b7efa84cf3459190ee4c8335b1ce7de5f19240666000462ddebcc0db45893"}
Oct 30 01:10:01.030958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.030935    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:10:01.044556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.044528    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"]
Oct 30 01:10:01.045335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.045290    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy" containerID="cri-o://c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f" gracePeriod=30
Oct 30 01:10:01.066915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.066882    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:01.164306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.156009    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq" podStartSLOduration=2.736984558 podStartE2EDuration="3.155986051s" totalImagesPullingTime="419.001493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:01.154784177 +0000 UTC m=+1709.184094417" watchObservedRunningTime="2026-10-30 01:10:01.155986051 +0000 UTC m=+1709.185296291"
Oct 30 01:10:01.232890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.231469    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:01.314328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.314123    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"]
Oct 30 01:10:01.314523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.314362    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy" containerID="cri-o://e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a" gracePeriod=30
Oct 30 01:10:01.506919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.479910    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:01.565868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.565357    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 01:10:01.565868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:01.565420    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 01:10:02.565870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:02.565256    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 01:10:02.565870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:02.565328    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 01:10:03.603494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:03.603458    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:10:03.604736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:03.604715    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:10:03.652033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:03.651988    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:04.952826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:04.952785    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:05.555064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:05.555030    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"]
Oct 30 01:10:05.555279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:05.555230    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy" containerID="cri-o://3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc" gracePeriod=30
Oct 30 01:10:05.639527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:05.639489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:06.416493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.416470    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:10:06.429222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.429158    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:10:06.451999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.451972    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.452514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452421    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.452514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452474    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.452514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452509    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.452706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452565    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.452706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452615    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.452706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452650    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.453266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452730    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.453266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452793    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.453266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452961    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.453266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.452998    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.453266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.453034    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.453084    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.454437    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.454551    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.454615    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.454670    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token\") pod \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\" (UID: \"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.454708    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo\") pod \"73cf2e06-7295-4028-9c9c-d140fe5d7147\" (UID: \"73cf2e06-7295-4028-9c9c-d140fe5d7147\") "
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.455186    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.458016    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.458194    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.459011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.458782    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.459643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.459217    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.460661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.459780    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:06.460661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.460089    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.461890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.461870    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.463147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463126    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463151    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463165    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463178    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463196    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463209    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73cf2e06-7295-4028-9c9c-d140fe5d7147-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463224    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463239    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.463779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463366    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:06.463779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.463691    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "kube-api-access-6tppm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:06.465024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.464998    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.482002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.481966    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.489050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.489012    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.490119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.489908    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:06.490119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.490042    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt" pod "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" (UID: "6dd4ee60-0c06-496d-9ea7-aff8c189ecd4"). InnerVolumeSpecName "kube-api-access-jqxvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:06.490298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.490144    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:06.490830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.490748    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:06.492274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.492136    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy" pod "73cf2e06-7295-4028-9c9c-d140fe5d7147" (UID: "73cf2e06-7295-4028-9c9c-d140fe5d7147"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.568213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568170    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568214    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568228    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568240    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568254    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tppm\" (UniqueName: \"kubernetes.io/projected/73cf2e06-7295-4028-9c9c-d140fe5d7147-kube-api-access-6tppm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568271    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568283    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73cf2e06-7295-4028-9c9c-d140fe5d7147-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568295    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568307    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqxvt\" (UniqueName: \"kubernetes.io/projected/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4-kube-api-access-jqxvt\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.568462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.568321    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73cf2e06-7295-4028-9c9c-d140fe5d7147-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.625085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.625058    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:10:06.668949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.668918    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.668975    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669077    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669109    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669185    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669235    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669373    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.669527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669402    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669502    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669654    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket\") pod \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\" (UID: \"b10f3bf4-0a71-4aad-866f-4bf48958f6b7\") "
Oct 30 01:10:06.669733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.669711    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.670040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670017    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.670129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670037    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.670129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670044    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:06.670248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670208    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.670248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670222    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.670248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670231    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.670248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670239    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.670248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.670247    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.672164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.672144    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:06.672164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.672156    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:06.672525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.672486    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "kube-api-access-qpf2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:06.673234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.672650    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token" pod "b10f3bf4-0a71-4aad-866f-4bf48958f6b7" (UID: "b10f3bf4-0a71-4aad-866f-4bf48958f6b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:06.771690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.771655    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.771690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.771684    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.771690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.771692    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:06.771690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:06.771702    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpf2c\" (UniqueName: \"kubernetes.io/projected/b10f3bf4-0a71-4aad-866f-4bf48958f6b7-kube-api-access-qpf2c\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:07.043521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.043490    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:10:07.088222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.088179    4577 generic.go:356] "Generic (PLEG): container finished" podID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerID="e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a" exitCode=0
Oct 30 01:10:07.088394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.088341    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"
Oct 30 01:10:07.088884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.088839    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" event={"ID":"b10f3bf4-0a71-4aad-866f-4bf48958f6b7","Type":"ContainerDied","Data":"e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a"}
Oct 30 01:10:07.089005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.088919    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp" event={"ID":"b10f3bf4-0a71-4aad-866f-4bf48958f6b7","Type":"ContainerDied","Data":"938fd40a02170758d60696a597d0be729a73625ea532a7d5d49a474a5d71308f"}
Oct 30 01:10:07.089005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.088951    4577 scope.go:122] "RemoveContainer" containerID="e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a"
Oct 30 01:10:07.095429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.095393    4577 generic.go:356] "Generic (PLEG): container finished" podID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerID="39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e" exitCode=0
Oct 30 01:10:07.095538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.095457    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"
Oct 30 01:10:07.095612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.095548    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" event={"ID":"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4","Type":"ContainerDied","Data":"39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e"}
Oct 30 01:10:07.095612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.095597    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc" event={"ID":"6dd4ee60-0c06-496d-9ea7-aff8c189ecd4","Type":"ContainerDied","Data":"9c8c38cc0fd7984f05fe675cd8c9d79754f934c4e60dc554b6229ed44bca8e21"}
Oct 30 01:10:07.097956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.097339    4577 generic.go:356] "Generic (PLEG): container finished" podID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerID="c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f" exitCode=0
Oct 30 01:10:07.097956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.097400    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"
Oct 30 01:10:07.097956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.097611    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" event={"ID":"73cf2e06-7295-4028-9c9c-d140fe5d7147","Type":"ContainerDied","Data":"c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f"}
Oct 30 01:10:07.097956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.097652    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq" event={"ID":"73cf2e06-7295-4028-9c9c-d140fe5d7147","Type":"ContainerDied","Data":"bf9013df873a7ff0f25e7547cb45dbe57b44d12937f94395fb7a9cdecd771e35"}
Oct 30 01:10:07.112731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.112704    4577 scope.go:122] "RemoveContainer" containerID="37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"
Oct 30 01:10:07.127715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.127659    4577 scope.go:122] "RemoveContainer" containerID="e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a"
Oct 30 01:10:07.128208 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:07.128118    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a\": container with ID starting with e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a not found: ID does not exist" containerID="e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a"
Oct 30 01:10:07.128208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.128164    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a"} err="failed to get container status \"e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a\": rpc error: code = NotFound desc = could not find container \"e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a\": container with ID starting with e7265eeacabfc049868a425cce063178374acaa341ffc1eeb9af740d62a34c6a not found: ID does not exist"
Oct 30 01:10:07.128208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.128188    4577 scope.go:122] "RemoveContainer" containerID="37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"
Oct 30 01:10:07.128554 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:07.128526    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a\": container with ID starting with 37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a not found: ID does not exist" containerID="37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"
Oct 30 01:10:07.128807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.128565    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a"} err="failed to get container status \"37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a\": rpc error: code = NotFound desc = could not find container \"37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a\": container with ID starting with 37944d28ebcb593a68796c59d421189dc0a9e0bbdc28fd2a7d20c7fd9bd5269a not found: ID does not exist"
Oct 30 01:10:07.128807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.128585    4577 scope.go:122] "RemoveContainer" containerID="39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e"
Oct 30 01:10:07.142115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.142092    4577 scope.go:122] "RemoveContainer" containerID="39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e"
Oct 30 01:10:07.142385 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:07.142365    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e\": container with ID starting with 39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e not found: ID does not exist" containerID="39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e"
Oct 30 01:10:07.142469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.142396    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e"} err="failed to get container status \"39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e\": rpc error: code = NotFound desc = could not find container \"39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e\": container with ID starting with 39bfe81fb3af7090244a0d9180fb6f82ee172d52e593373e163d849d09568a1e not found: ID does not exist"
Oct 30 01:10:07.142469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.142415    4577 scope.go:122] "RemoveContainer" containerID="c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f"
Oct 30 01:10:07.160891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.155196    4577 scope.go:122] "RemoveContainer" containerID="c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f"
Oct 30 01:10:07.160891 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:07.155575    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f\": container with ID starting with c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f not found: ID does not exist" containerID="c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f"
Oct 30 01:10:07.160891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.155609    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f"} err="failed to get container status \"c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f\": rpc error: code = NotFound desc = could not find container \"c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f\": container with ID starting with c1f3f69205e75148935db546fcf45c84b9fc16ebf3a2fc70305087c0f1a4dd8f not found: ID does not exist"
Oct 30 01:10:07.177062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.176973    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"]
Oct 30 01:10:07.181327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.181283    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-a8ab86-openshift-default-59b8cfc94c-l2jzp"]
Oct 30 01:10:07.236343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.236313    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"]
Oct 30 01:10:07.239715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.239694    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-smbdq"]
Oct 30 01:10:07.277721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.277690    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"]
Oct 30 01:10:07.283734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.283709    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4rlhc"]
Oct 30 01:10:07.425892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.425780    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:07.797908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.797878    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:07.908499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:07.908466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:08.112458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:08.112387    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" path="/var/lib/kubelet/pods/6dd4ee60-0c06-496d-9ea7-aff8c189ecd4/volumes"
Oct 30 01:10:08.112862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:08.112833    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" path="/var/lib/kubelet/pods/73cf2e06-7295-4028-9c9c-d140fe5d7147/volumes"
Oct 30 01:10:08.113286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:08.113267    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" path="/var/lib/kubelet/pods/b10f3bf4-0a71-4aad-866f-4bf48958f6b7/volumes"
Oct 30 01:10:08.812569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:08.812541    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:08.929041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:08.929010    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:09.178824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:09.178703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:09.940043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:09.940004    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:10:09.940552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:09.940090    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:10:11.021437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.021213    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:11.081946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.081896    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 01:10:11.148142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.148106    4577 generic.go:356] "Generic (PLEG): container finished" podID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerID="3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc" exitCode=0
Oct 30 01:10:11.148596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.148447    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"
Oct 30 01:10:11.148906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.148467    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" event={"ID":"88be2916-cb08-4316-9ca4-f4a4bd0cfeca","Type":"ContainerDied","Data":"3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc"}
Oct 30 01:10:11.149063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.149048    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d" event={"ID":"88be2916-cb08-4316-9ca4-f4a4bd0cfeca","Type":"ContainerDied","Data":"1b51e8fbc2ef097d346e4ba446ec71bb5ae73ea98892784f73f8198376412883"}
Oct 30 01:10:11.149186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.149175    4577 scope.go:122] "RemoveContainer" containerID="3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc"
Oct 30 01:10:11.179067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.179036    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.179349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.179328    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.179613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.179593    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.179812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.179797    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.182390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.182355    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.182623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.182608    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.182788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.182774    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.182963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.182935    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.183111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.183096    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6\") pod \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\" (UID: \"88be2916-cb08-4316-9ca4-f4a4bd0cfeca\") "
Oct 30 01:10:11.183349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.180253    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:11.183471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.180520    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:11.183870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.183829    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.183977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.183967    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.184056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.183001    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:11.184110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.183131    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:11.184617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.184316    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:11.185222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.185189    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:11.185222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.185622    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:11.185222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.190215    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "kube-api-access-ms2g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:11.214212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.199963    4577 scope.go:122] "RemoveContainer" containerID="b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"
Oct 30 01:10:11.223899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.222152    4577 scope.go:122] "RemoveContainer" containerID="3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc"
Oct 30 01:10:11.223899 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:11.222514    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc\": container with ID starting with 3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc not found: ID does not exist" containerID="3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc"
Oct 30 01:10:11.223899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.222555    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc"} err="failed to get container status \"3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc\": rpc error: code = NotFound desc = could not find container \"3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc\": container with ID starting with 3ee0aca7e0c9ffaa21265751313abd94a6bda9bf7c2c733c642649a0b17d0ffc not found: ID does not exist"
Oct 30 01:10:11.223899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.222579    4577 scope.go:122] "RemoveContainer" containerID="b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"
Oct 30 01:10:11.223899 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:11.223228    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa\": container with ID starting with b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa not found: ID does not exist" containerID="b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"
Oct 30 01:10:11.223899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.223266    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa"} err="failed to get container status \"b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa\": rpc error: code = NotFound desc = could not find container \"b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa\": container with ID starting with b5a7d9a5caab4f9a4d3b5addefe004968c19929c0d755b2e4a763930d36558fa not found: ID does not exist"
Oct 30 01:10:11.226788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.226757    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token" pod "88be2916-cb08-4316-9ca4-f4a4bd0cfeca" (UID: "88be2916-cb08-4316-9ca4-f4a4bd0cfeca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:11.285959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285917    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.285959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285947    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.285959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285957    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms2g6\" (UniqueName: \"kubernetes.io/projected/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-kube-api-access-ms2g6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.285959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285966    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.285959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285974    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.286395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285982    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.286395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.285991    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88be2916-cb08-4316-9ca4-f4a4bd0cfeca-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:11.520226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.520204    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"]
Oct 30 01:10:11.531117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:11.531084    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-zjl7d"]
Oct 30 01:10:12.113207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:12.113178    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" path="/var/lib/kubelet/pods/88be2916-cb08-4316-9ca4-f4a4bd0cfeca/volumes"
Oct 30 01:10:12.980010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:12.979979    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:14.380279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:14.380239    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:14.862306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:14.862273    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"]
Oct 30 01:10:14.862835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:14.862797    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" containerID="cri-o://2da86ce71dfacf13a0cdd4627dbdd1a75bff3a8f0de05d2b550187eb6fdbca42" gracePeriod=30
Oct 30 01:10:14.982876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:14.973531    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:15.559743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:15.559708    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:16.004514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.004486    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"]
Oct 30 01:10:16.005156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005142    4577 cpu_manager.go:414] "containerMap: removing container" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy"
Oct 30 01:10:16.005208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005158    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy"
Oct 30 01:10:16.005208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005169    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy"
Oct 30 01:10:16.005208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005175    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy"
Oct 30 01:10:16.005208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005206    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005211    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005223    4577 cpu_manager.go:414] "containerMap: removing container" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005228    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005241    4577 cpu_manager.go:414] "containerMap: removing container" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005245    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005254    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005258    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005349    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy"
Oct 30 01:10:16.005360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005359    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy"
Oct 30 01:10:16.005623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005369    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="88be2916-cb08-4316-9ca4-f4a4bd0cfeca" containerName="istio-proxy"
Oct 30 01:10:16.005623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005375    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6dd4ee60-0c06-496d-9ea7-aff8c189ecd4" containerName="istio-proxy"
Oct 30 01:10:16.005623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005382    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="73cf2e06-7295-4028-9c9c-d140fe5d7147" containerName="istio-proxy"
Oct 30 01:10:16.005623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.005387    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b10f3bf4-0a71-4aad-866f-4bf48958f6b7" containerName="istio-proxy"
Oct 30 01:10:16.010763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.010737    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.044562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.044530    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"]
Oct 30 01:10:16.058718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.058644    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-975807-openshift-default-dockercfg-pbxt4\""
Oct 30 01:10:16.102838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.102798    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.102910    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.102976    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.103107    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.103142    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.103195    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.103278    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhstw\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.103317    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.103373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.103342    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.115703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.115674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.216969    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217061    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217124    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217213    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhstw\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217254    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217280    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217359    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217404    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.219265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.217443    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.225879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.223362    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.268317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.268237    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.268899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.268876    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.277867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.269609    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.289601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.288473    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.289601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.288726    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.289601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.289326    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.303089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.303029    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.330376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.329584    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhstw\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw\") pod \"e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") " pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.341060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.340770    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:16.524190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.524056    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:16.526112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.526090    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"]
Oct 30 01:10:16.526647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.526310    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl" podUID="e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" containerName="maas-api" containerID="cri-o://799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7" gracePeriod=30
Oct 30 01:10:16.694509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.692220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:16.695564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.695528    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"]
Oct 30 01:10:16.695761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.695739    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz" podUID="f87e08bb-d73e-4702-8d8e-92be5ae8190f" containerName="payload-processing" containerID="cri-o://66ba403cecf598b25be9cb79bfd8cb22a664fde1db4182d5d0858276620cab95" gracePeriod=30
Oct 30 01:10:16.700653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.700625    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"]
Oct 30 01:10:16.700836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.700816    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc" podUID="fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" containerName="payload-pre-processing" containerID="cri-o://f076d1d52df7167d46da8f74fd0d16ac0ae6a666312aff27fbc8742cc898516b" gracePeriod=30
Oct 30 01:10:16.813664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.813625    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"]
Oct 30 01:10:16.825972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.825387    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:16.837384 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:16.836295    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee84bb2_ddc3_49b9_a772_ffeef3ffc820.slice/crio-39dc018039c074aa96fd22be4815ea556167e944ed7ca83bd6708268fbea14bc WatchSource:0}: Error finding container 39dc018039c074aa96fd22be4815ea556167e944ed7ca83bd6708268fbea14bc: Status 404 returned error can't find the container with id 39dc018039c074aa96fd22be4815ea556167e944ed7ca83bd6708268fbea14bc
Oct 30 01:10:16.869472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.844652    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:16.869472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.844735    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:16.869472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:16.844775    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:17.292408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.292076    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:10:17.299329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.299298    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:17.306410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.306379    4577 generic.go:356] "Generic (PLEG): container finished" podID="e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" containerID="799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7" exitCode=0
Oct 30 01:10:17.306553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.306530    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl" event={"ID":"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1","Type":"ContainerDied","Data":"799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7"}
Oct 30 01:10:17.306630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.306594    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl" event={"ID":"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1","Type":"ContainerDied","Data":"7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c"}
Oct 30 01:10:17.306630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.306624    4577 scope.go:122] "RemoveContainer" containerID="799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7"
Oct 30 01:10:17.307455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.307342    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"
Oct 30 01:10:17.312948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.312919    4577 generic.go:356] "Generic (PLEG): container finished" podID="f87e08bb-d73e-4702-8d8e-92be5ae8190f" containerID="66ba403cecf598b25be9cb79bfd8cb22a664fde1db4182d5d0858276620cab95" exitCode=0
Oct 30 01:10:17.313117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.313040    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz" event={"ID":"f87e08bb-d73e-4702-8d8e-92be5ae8190f","Type":"ContainerDied","Data":"66ba403cecf598b25be9cb79bfd8cb22a664fde1db4182d5d0858276620cab95"}
Oct 30 01:10:17.315691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.315648    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" event={"ID":"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820","Type":"ContainerStarted","Data":"39dc018039c074aa96fd22be4815ea556167e944ed7ca83bd6708268fbea14bc"}
Oct 30 01:10:17.319714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.319525    4577 generic.go:356] "Generic (PLEG): container finished" podID="fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" containerID="f076d1d52df7167d46da8f74fd0d16ac0ae6a666312aff27fbc8742cc898516b" exitCode=0
Oct 30 01:10:17.319714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.319618    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc" event={"ID":"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb","Type":"ContainerDied","Data":"f076d1d52df7167d46da8f74fd0d16ac0ae6a666312aff27fbc8742cc898516b"}
Oct 30 01:10:17.402404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.402375    4577 scope.go:122] "RemoveContainer" containerID="799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7"
Oct 30 01:10:17.403145 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:17.403113    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7\": container with ID starting with 799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7 not found: ID does not exist" containerID="799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7"
Oct 30 01:10:17.403272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.403158    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7"} err="failed to get container status \"799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7\": rpc error: code = NotFound desc = could not find container \"799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7\": container with ID starting with 799b49b464c1c9b1801ece21bbcef0947075ed01a56b7e5c5747bb86e72452c7 not found: ID does not exist"
Oct 30 01:10:17.405287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.405256    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca\") pod \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") "
Oct 30 01:10:17.405419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.405312    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs\") pod \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") "
Oct 30 01:10:17.405479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.405464    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn\" (UniqueName: \"kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn\") pod \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") "
Oct 30 01:10:17.405637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.405616    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls\") pod \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\" (UID: \"e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1\") "
Oct 30 01:10:17.408206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.407564    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:17.408336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.408225    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca" pod "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" (UID: "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:17.409576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.409553    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs" pod "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" (UID: "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:10:17.411283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.411258    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:10:17.411929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.411908    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn" pod "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" (UID: "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1"). InnerVolumeSpecName "kube-api-access-mhhjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:17.413414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.413391    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls" pod "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" (UID: "e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:10:17.430011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.429886    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:17.437923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.437074    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podStartSLOduration=2.437056093 podStartE2EDuration="2.437056093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:17.434280298 +0000 UTC m=+1725.463590551" watchObservedRunningTime="2026-10-30 01:10:17.437056093 +0000 UTC m=+1725.466366327"
Oct 30 01:10:17.509724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.509696    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.509931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.509918    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.510005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.509996    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.510086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.510076    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhhjn\" (UniqueName: \"kubernetes.io/projected/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1-kube-api-access-mhhjn\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.513063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.513043    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:10:17.708007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.666254    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume\") pod \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") "
Oct 30 01:10:17.711278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.711234    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf\" (UniqueName: \"kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf\") pod \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\" (UID: \"f87e08bb-d73e-4702-8d8e-92be5ae8190f\") "
Oct 30 01:10:17.717828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.714610    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume" pod "f87e08bb-d73e-4702-8d8e-92be5ae8190f" (UID: "f87e08bb-d73e-4702-8d8e-92be5ae8190f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:17.717828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.717009    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:17.717828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.717624    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87e08bb-d73e-4702-8d8e-92be5ae8190f-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.722104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.722041    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf" pod "f87e08bb-d73e-4702-8d8e-92be5ae8190f" (UID: "f87e08bb-d73e-4702-8d8e-92be5ae8190f"). InnerVolumeSpecName "kube-api-access-nwpkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:17.775257 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:17.753056    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37ccf5c_5855_4b20_9b6c_f6ff3d5e98b1.slice/crio-7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c\": RecentStats: unable to find data in memory cache]"
Oct 30 01:10:17.775257 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:17.770985    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37ccf5c_5855_4b20_9b6c_f6ff3d5e98b1.slice/crio-7f8fc599ff625ab105c5a94bc325d9a24be4ea03db612fe01d71fd7e79214e9c\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37ccf5c_5855_4b20_9b6c_f6ff3d5e98b1.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 01:10:17.818088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.818051    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l\" (UniqueName: \"kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l\") pod \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") "
Oct 30 01:10:17.818429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.818394    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume\") pod \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\" (UID: \"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb\") "
Oct 30 01:10:17.818965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.818929    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwpkf\" (UniqueName: \"kubernetes.io/projected/f87e08bb-d73e-4702-8d8e-92be5ae8190f-kube-api-access-nwpkf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.819441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.819388    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume" pod "fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" (UID: "fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:17.821923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.821901    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l" pod "fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" (UID: "fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb"). InnerVolumeSpecName "kube-api-access-frm2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:17.823137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.823110    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm_25441484-890b-4e6c-94e0-b4dd99376b8c/payload-processing/0.log"
Oct 30 01:10:17.834626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.834597    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"]
Oct 30 01:10:17.843424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.843396    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-429c67-68f96796cd-64jjl"]
Oct 30 01:10:17.911737 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.911665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:17.927664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.927625    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:17.927664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:17.927665    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frm2l\" (UniqueName: \"kubernetes.io/projected/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb-kube-api-access-frm2l\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:18.113452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.113415    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" path="/var/lib/kubelet/pods/e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1/volumes"
Oct 30 01:10:18.284200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.284162    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:10:18.284451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.284416    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:10:18.331765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.331731    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"
Oct 30 01:10:18.331765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.331747    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz" event={"ID":"f87e08bb-d73e-4702-8d8e-92be5ae8190f","Type":"ContainerDied","Data":"cdbfbf976d04d8feee73eefd3361c6b401111117e49c7943640814548c1a6d1a"}
Oct 30 01:10:18.332049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.331826    4577 scope.go:122] "RemoveContainer" containerID="66ba403cecf598b25be9cb79bfd8cb22a664fde1db4182d5d0858276620cab95"
Oct 30 01:10:18.336033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.335261    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" event={"ID":"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820","Type":"ContainerStarted","Data":"69a8635b3e117b242f4268f749c23e9cf2786c92850dcffa4bf0dee76e8b1525"}
Oct 30 01:10:18.337660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.337638    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc" event={"ID":"fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb","Type":"ContainerDied","Data":"aa2c43e08b23172ab7a67d547681efcc7108c30aa257a49740e4661df5ba2a01"}
Oct 30 01:10:18.337772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.337668    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"
Oct 30 01:10:18.345768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.345745    4577 scope.go:122] "RemoveContainer" containerID="f076d1d52df7167d46da8f74fd0d16ac0ae6a666312aff27fbc8742cc898516b"
Oct 30 01:10:18.348123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.348094    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:18.442268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.442235    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"]
Oct 30 01:10:18.447974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.447948    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-429c67-5fc5766d67-f4fbc"]
Oct 30 01:10:18.492663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.492580    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"]
Oct 30 01:10:18.498092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:18.498059    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-429c67-6c55b64db-zbmlz"]
Oct 30 01:10:19.358140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.358090    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:19.359375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.359351    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:10:19.500667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.500636    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"]
Oct 30 01:10:19.501524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501505    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" containerName="payload-pre-processing"
Oct 30 01:10:19.501524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501522    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" containerName="payload-pre-processing"
Oct 30 01:10:19.501645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501537    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f87e08bb-d73e-4702-8d8e-92be5ae8190f" containerName="payload-processing"
Oct 30 01:10:19.501645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501548    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f87e08bb-d73e-4702-8d8e-92be5ae8190f" containerName="payload-processing"
Oct 30 01:10:19.501645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501580    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" containerName="maas-api"
Oct 30 01:10:19.501645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501586    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" containerName="maas-api"
Oct 30 01:10:19.501774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501712    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e37ccf5c-5855-4b20-9b6c-f6ff3d5e98b1" containerName="maas-api"
Oct 30 01:10:19.501774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501731    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" containerName="payload-pre-processing"
Oct 30 01:10:19.501774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.501744    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f87e08bb-d73e-4702-8d8e-92be5ae8190f" containerName="payload-processing"
Oct 30 01:10:19.507214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.507189    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.527742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.524971    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f05249\"/\"openshift-service-ca.crt\""
Oct 30 01:10:19.527742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.525680    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f05249\"/\"ipp-route-f05249-kserve-self-signed-certs\""
Oct 30 01:10:19.527742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.525791    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f05249\"/\"default-dockercfg-qxjcw\""
Oct 30 01:10:19.528247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.528213    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f05249\"/\"kube-root-ca.crt\""
Oct 30 01:10:19.532316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.532282    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"]
Oct 30 01:10:19.553822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.553317    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.553822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.553499    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.553822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.553565    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.553822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.553643    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.553822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.553718    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8vw9\" (UniqueName: \"kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.553822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.553765    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.554952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.554922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:19.655177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655082    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655192    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655268    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8vw9\" (UniqueName: \"kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655308    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655341    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655510    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655556    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655569    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.655899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.655879    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.658478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.658454    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.658566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.658511    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.681188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.681154    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8vw9\" (UniqueName: \"kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9\") pod \"ipp-route-f05249-kserve-74fdc7f7-7gl8w\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") " pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.826870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.825286    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:19.835370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.835335    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:19.972346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:19.963392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:20.112053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.112020    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f87e08bb-d73e-4702-8d8e-92be5ae8190f" path="/var/lib/kubelet/pods/f87e08bb-d73e-4702-8d8e-92be5ae8190f/volumes"
Oct 30 01:10:20.112537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.112517    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb" path="/var/lib/kubelet/pods/fd2b51c6-2d0e-4d6a-b1c8-93dd4a9524eb/volumes"
Oct 30 01:10:20.219053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.219027    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"]
Oct 30 01:10:20.219621 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:20.219591    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e575dc9_198f_4329_b2b0_54ade58440fa.slice/crio-1c67b2aabc4231d96ff38f80293a8138104514e297581c081f927f9894ed5e21 WatchSource:0}: Error finding container 1c67b2aabc4231d96ff38f80293a8138104514e297581c081f927f9894ed5e21: Status 404 returned error can't find the container with id 1c67b2aabc4231d96ff38f80293a8138104514e297581c081f927f9894ed5e21
Oct 30 01:10:20.371343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.371052    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" event={"ID":"2e575dc9-198f-4329-b2b0-54ade58440fa","Type":"ContainerStarted","Data":"11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0"}
Oct 30 01:10:20.371343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.371114    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" event={"ID":"2e575dc9-198f-4329-b2b0-54ade58440fa","Type":"ContainerStarted","Data":"1c67b2aabc4231d96ff38f80293a8138104514e297581c081f927f9894ed5e21"}
Oct 30 01:10:20.373820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.373788    4577 generic.go:356] "Generic (PLEG): container finished" podID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerID="2da86ce71dfacf13a0cdd4627dbdd1a75bff3a8f0de05d2b550187eb6fdbca42" exitCode=0
Oct 30 01:10:20.374306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.374282    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" event={"ID":"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105","Type":"ContainerDied","Data":"2da86ce71dfacf13a0cdd4627dbdd1a75bff3a8f0de05d2b550187eb6fdbca42"}
Oct 30 01:10:20.374504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.374480    4577 scope.go:122] "RemoveContainer" containerID="d31e949ee49dc6e7b5a13b19424c9a25086408711b3062cfe889600b825089d9"
Oct 30 01:10:20.421254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.421193    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" podStartSLOduration=1.4211741070000001 podStartE2EDuration="1.421174107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:20.418813035 +0000 UTC m=+1728.448123288" watchObservedRunningTime="2026-10-30 01:10:20.421174107 +0000 UTC m=+1728.450484341"
Oct 30 01:10:20.478574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.472031    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:10:20.478574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.472112    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:10:20.597272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.597241    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 01:10:20.692758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.692707    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:20.697088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697059    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697132    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697198    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697228    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697313    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697346    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697387    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697413    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.697636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.697566    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo\") pod \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\" (UID: \"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105\") "
Oct 30 01:10:20.698266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.698241    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:20.698443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.698398    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:20.698614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.698587    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:20.698812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.698791    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:20.698934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.698816    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:20.701514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.701482    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:20.702422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.702397    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "kube-api-access-cts9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:20.704141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.704118    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:20.704392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.704362    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo" pod "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" (UID: "6c09b581-ccc6-46f8-8bf8-95cb0c3bc105"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824071    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824110    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cts9k\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-kube-api-access-cts9k\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824126    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824141    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824153    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824165    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824177    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824189    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.824229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.824200    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:20.843865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.832353    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:20.845363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:20.844905    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:21.211195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.211113    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:21.386350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.386315    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml" event={"ID":"6c09b581-ccc6-46f8-8bf8-95cb0c3bc105","Type":"ContainerDied","Data":"0e1002811da34baf1a7d3633e6c0db96ab917336eefb2912300a51f24fc752e1"}
Oct 30 01:10:21.386350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.386353    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"
Oct 30 01:10:21.387085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.386387    4577 scope.go:122] "RemoveContainer" containerID="2da86ce71dfacf13a0cdd4627dbdd1a75bff3a8f0de05d2b550187eb6fdbca42"
Oct 30 01:10:21.422153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.422112    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:21.480417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.480378    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"]
Oct 30 01:10:21.489224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.489182    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-9x2ml"]
Oct 30 01:10:21.596352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.596311    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:21.718002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.717960    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:21.752197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.752102    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"]
Oct 30 01:10:21.752723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.752704    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy"
Oct 30 01:10:21.752723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.752721    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy"
Oct 30 01:10:21.752935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.752872    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy"
Oct 30 01:10:21.757723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.757694    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:21.782261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.782182    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-975807-dockercfg-tc58p\""
Oct 30 01:10:21.782417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.782328    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-975807\""
Oct 30 01:10:21.785954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.785924    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"]
Oct 30 01:10:21.863872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.863695    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:21.863872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.863749    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fshk\" (UniqueName: \"kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk\") pod \"payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:21.890866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.889901    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"]
Oct 30 01:10:21.891039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.890908    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy"
Oct 30 01:10:21.891039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.890928    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy"
Oct 30 01:10:21.891142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.891105    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" containerName="istio-proxy"
Oct 30 01:10:21.989083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.972574    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:21.989083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.972629    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fshk\" (UniqueName: \"kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk\") pod \"payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:21.989083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.983784    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.000872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:21.990014    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:22.022001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.011651    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"]
Oct 30 01:10:22.025776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.025746    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fshk\" (UniqueName: \"kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk\") pod \"payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:22.074668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.074634    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b97f9\" (UniqueName: \"kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9\") pod \"payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.074837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.074693    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.077112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.077093    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:22.115504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.115349    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c09b581-ccc6-46f8-8bf8-95cb0c3bc105" path="/var/lib/kubelet/pods/6c09b581-ccc6-46f8-8bf8-95cb0c3bc105/volumes"
Oct 30 01:10:22.175502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.175469    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b97f9\" (UniqueName: \"kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9\") pod \"payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.177102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.175591    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.177102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.176450    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.206235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.201309    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b97f9\" (UniqueName: \"kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9\") pod \"payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.331062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.331036    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:22.387233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.387196    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"]
Oct 30 01:10:22.390021 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:22.389992    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fea24a_068c_438a_a7b9_d1be036ce3ec.slice/crio-8efacc4838dcf4d9490f909feaa9d8b8376f6ac907c2ffa06a9af8f70a9badbb WatchSource:0}: Error finding container 8efacc4838dcf4d9490f909feaa9d8b8376f6ac907c2ffa06a9af8f70a9badbb: Status 404 returned error can't find the container with id 8efacc4838dcf4d9490f909feaa9d8b8376f6ac907c2ffa06a9af8f70a9badbb
Oct 30 01:10:22.404180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.401061    4577 generic.go:356] "Generic (PLEG): container finished" podID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerID="13321903ceee490d5a1bc4039a0bd595312560850b81d3c17c162d0f582b2052" exitCode=0
Oct 30 01:10:22.404180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.401099    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" event={"ID":"c1a46170-e759-428e-aa74-dbf88c1fdd26","Type":"ContainerDied","Data":"13321903ceee490d5a1bc4039a0bd595312560850b81d3c17c162d0f582b2052"}
Oct 30 01:10:22.404180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.401562    4577 scope.go:122] "RemoveContainer" containerID="13321903ceee490d5a1bc4039a0bd595312560850b81d3c17c162d0f582b2052"
Oct 30 01:10:22.411120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.411015    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:22.411120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.411115    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:22.411313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.411161    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:22.637097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.636983    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:22.639557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:22.639528    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"]
Oct 30 01:10:22.640678 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:22.640621    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975c8a5a_790b_4560_8d0d_feaf1a7155be.slice/crio-8dc83a4e466d548ed63754b5951b78584b908d5700a16d911f810a379ddec2b4 WatchSource:0}: Error finding container 8dc83a4e466d548ed63754b5951b78584b908d5700a16d911f810a379ddec2b4: Status 404 returned error can't find the container with id 8dc83a4e466d548ed63754b5951b78584b908d5700a16d911f810a379ddec2b4
Oct 30 01:10:23.410221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.410187    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" event={"ID":"c1a46170-e759-428e-aa74-dbf88c1fdd26","Type":"ContainerStarted","Data":"5f4c28e3754229d9a3646e0363609688b96beda080c069b38dce295be6920c9b"}
Oct 30 01:10:23.412002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.411974    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr" event={"ID":"78fea24a-068c-438a-a7b9-d1be036ce3ec","Type":"ContainerStarted","Data":"6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696"}
Oct 30 01:10:23.412135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.412023    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:23.412135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.412039    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr" event={"ID":"78fea24a-068c-438a-a7b9-d1be036ce3ec","Type":"ContainerStarted","Data":"8efacc4838dcf4d9490f909feaa9d8b8376f6ac907c2ffa06a9af8f70a9badbb"}
Oct 30 01:10:23.413671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.413649    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d" event={"ID":"975c8a5a-790b-4560-8d0d-feaf1a7155be","Type":"ContainerStarted","Data":"f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0"}
Oct 30 01:10:23.413771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.413687    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d" event={"ID":"975c8a5a-790b-4560-8d0d-feaf1a7155be","Type":"ContainerStarted","Data":"8dc83a4e466d548ed63754b5951b78584b908d5700a16d911f810a379ddec2b4"}
Oct 30 01:10:23.413771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.413735    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:23.522605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.522551    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr" podStartSLOduration=2.522536633 podStartE2EDuration="2.522536633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:23.519929809 +0000 UTC m=+1731.549240041" watchObservedRunningTime="2026-10-30 01:10:23.522536633 +0000 UTC m=+1731.551846864"
Oct 30 01:10:23.567485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.567425    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d" podStartSLOduration=2.5673899860000002 podStartE2EDuration="2.567389986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:23.565367572 +0000 UTC m=+1731.594677805" watchObservedRunningTime="2026-10-30 01:10:23.567389986 +0000 UTC m=+1731.596700219"
Oct 30 01:10:23.613373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.613338    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:10:23.613537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.613487    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 01:10:23.613581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:23.613529    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 01:10:24.597760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:24.597728    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:24.613987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:24.613959    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 01:10:24.614139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:24.614034    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 01:10:24.881673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:24.881573    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:25.017466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.017438    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:25.207162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.207072    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"]
Oct 30 01:10:25.207976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.207943    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy" containerID="cri-o://1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8" gracePeriod=30
Oct 30 01:10:25.224367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.224335    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:25.613537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.613505    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 01:10:25.613974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.613574    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 01:10:25.727096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:25.727052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:26.613468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:26.613437    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 01:10:26.613632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:26.613488    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 01:10:26.720089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:26.720058    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:26.906967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:26.906879    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:27.017419 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:27.017382    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:10:27.227263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.227181    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:27.234958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.234931    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"]
Oct 30 01:10:27.239486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.239465    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.280884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.268386    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-5f8cbc-openshift-default-dockercfg-zhpsn\""
Oct 30 01:10:27.294698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.294654    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"]
Oct 30 01:10:27.383727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.383436    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"]
Oct 30 01:10:27.391046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.391016    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:27.400143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.399107    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"]
Oct 30 01:10:27.400143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.399224    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.401862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.401820    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.401973    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402054    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402143    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402191    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402264    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402310    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r722h\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402350    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.402494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.402474    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.416417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.416383    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-88a39d-openshift-default-dockercfg-586fr\""
Oct 30 01:10:27.503820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.503784    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.504056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.503831    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.504056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.503899    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.503961    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504024    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504057    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.504281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504200    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504233    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504287    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.504458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504344    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504394    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r722h\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504393    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.504458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504452    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504479    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9k58\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504511    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504638    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504643    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504742    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504783    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504813    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.504931    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.505025    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.505211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.505085    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.506524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.506501    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:27.507279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.507262    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.509885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.509864    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.572873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.568591    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r722h\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.593949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.585275    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token\") pod \"e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") " pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.615163    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616268    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616318    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616340    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616395    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616424    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616465    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616522    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616593    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9k58\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.616619    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.617424    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.617626    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.617817    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.618684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.618391    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.626889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.626457    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 01:10:27.626889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.626514    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 01:10:27.633824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.633780    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.754818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.740228    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.754818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.740658    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.754818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.741162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:27.767034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:27.766967    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9k58\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58\") pod \"e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") " pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:28.020225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.020128    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:28.189212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.145816    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"]
Oct 30 01:10:28.193787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.193742    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:28.193963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.193835    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:28.193963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.193957    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:28.392724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.392522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:28.402898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.402839    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:28.403016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.402971    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:28.403083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.403023    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:10:28.404390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.404374    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"]
Oct 30 01:10:28.509599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.509154    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" event={"ID":"0464e965-6efa-49ae-a42d-900029d35dbc","Type":"ContainerStarted","Data":"44d5b90e8e2a502917a6df9dd9bc7a6af90c407fa3fd4ab3f3af4f8c0e5d9b2f"}
Oct 30 01:10:28.512245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.512170    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:28.516919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.515809    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" event={"ID":"e08d7191-2e5f-4812-b832-669b3aedaac8","Type":"ContainerStarted","Data":"e3c34069a2e2ee83bc5c4ac3a446490c7ef0e921aea7f7001faf1b7fa50df31a"}
Oct 30 01:10:28.516919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.515881    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" event={"ID":"e08d7191-2e5f-4812-b832-669b3aedaac8","Type":"ContainerStarted","Data":"c0e4bb130ad14e1f7f9ef96ad658b847c3e7db63d879046af105d1a6a8a13988"}
Oct 30 01:10:28.579922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.579301    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podStartSLOduration=1.579281003 podStartE2EDuration="1.579281003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:28.578426685 +0000 UTC m=+1736.607736920" watchObservedRunningTime="2026-10-30 01:10:28.579281003 +0000 UTC m=+1736.608591235"
Oct 30 01:10:28.613536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.613497    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused" start-of-body=
Oct 30 01:10:28.613825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.613796    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.169:15021/healthz/ready\": dial tcp 10.134.0.169:15021: connect: connection refused"
Oct 30 01:10:28.617630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.617602    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:28.626565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.626502    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:28.922314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:28.922220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:29.211078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.210984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:29.525622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.525584    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" event={"ID":"0464e965-6efa-49ae-a42d-900029d35dbc","Type":"ContainerStarted","Data":"99168b2a20b3f8d1db384c023678394e70b301cb65b85bf1d734a4d532fd0e10"}
Oct 30 01:10:29.577509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.577449    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podStartSLOduration=2.577435038 podStartE2EDuration="2.577435038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:29.573994656 +0000 UTC m=+1737.603304901" watchObservedRunningTime="2026-10-30 01:10:29.577435038 +0000 UTC m=+1737.606745348"
Oct 30 01:10:29.618458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.618426    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:10:29.618904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.618875    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:10:29.619714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.619690    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:10:29.624369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.624346    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:29.835973 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.835872    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:29.853617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:29.853593    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:30.021613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.021580    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:30.028801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.028777    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:30.504855 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.504812    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:10:30.535445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.535415    4577 generic.go:356] "Generic (PLEG): container finished" podID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerID="1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8" exitCode=0
Oct 30 01:10:30.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.535478    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"
Oct 30 01:10:30.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.535498    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" event={"ID":"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0","Type":"ContainerDied","Data":"1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8"}
Oct 30 01:10:30.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.535554    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6" event={"ID":"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0","Type":"ContainerDied","Data":"5f63c8b3e8bc9a089fd8b0b8b308780fda263c5b0a0401c54664ecd47aae7f3a"}
Oct 30 01:10:30.535601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.535581    4577 scope.go:122] "RemoveContainer" containerID="1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8"
Oct 30 01:10:30.535968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.535951    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:30.536661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.536531    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:30.537294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.537276    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:10:30.537584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.537559    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:10:30.542378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542351    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542414    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542464    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542501    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542538    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542569    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542639    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542703    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.542984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.542928    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs\") pod \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\" (UID: \"b19927d1-bfcb-4d1f-8a1d-79e9beed85f0\") "
Oct 30 01:10:30.543629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.543605    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:30.543791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.543641    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:30.543944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.543825    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:30.543944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.543741    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:30.544110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.544021    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:30.545987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.545961    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "kube-api-access-tlhs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:30.546322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.546300    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:30.546522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.546497    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:30.547813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.547790    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy" pod "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" (UID: "b19927d1-bfcb-4d1f-8a1d-79e9beed85f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:30.549623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.549597    4577 scope.go:122] "RemoveContainer" containerID="efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"
Oct 30 01:10:30.551002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.550983    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:10:30.574289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.574268    4577 scope.go:122] "RemoveContainer" containerID="1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8"
Oct 30 01:10:30.574617 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:30.574597    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8\": container with ID starting with 1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8 not found: ID does not exist" containerID="1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8"
Oct 30 01:10:30.574675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.574630    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8"} err="failed to get container status \"1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8\": rpc error: code = NotFound desc = could not find container \"1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8\": container with ID starting with 1d46ddba419e8dba156fd34069728ece73ba19c46d6d4d84489a9d91e9489bf8 not found: ID does not exist"
Oct 30 01:10:30.574675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.574649    4577 scope.go:122] "RemoveContainer" containerID="efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"
Oct 30 01:10:30.574923 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:30.574907    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b\": container with ID starting with efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b not found: ID does not exist" containerID="efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"
Oct 30 01:10:30.574985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.574928    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b"} err="failed to get container status \"efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b\": rpc error: code = NotFound desc = could not find container \"efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b\": container with ID starting with efa894a2ee26a7913e8a71cc130642568e26522d28b52ee7f3bcd0c07f47e24b not found: ID does not exist"
Oct 30 01:10:30.645701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645664    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645696    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645714    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645730    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645743    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645758    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlhs6\" (UniqueName: \"kubernetes.io/projected/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-kube-api-access-tlhs6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645771    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645791    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.645947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.645800    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:30.899371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.899328    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"]
Oct 30 01:10:30.905103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:30.905069    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-429c67-openshift-default-5b857ccd8c-7wtp6"]
Oct 30 01:10:31.100481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:31.100400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:31.218655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:31.218624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:31.967262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:31.967233    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:32.099601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:32.099570    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:32.123557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:32.123525    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" path="/var/lib/kubelet/pods/b19927d1-bfcb-4d1f-8a1d-79e9beed85f0/volumes"
Oct 30 01:10:32.805801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:32.805759    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:32.932182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:32.932144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:33.325292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:33.325253    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:34.379997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:34.378198    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:34.425334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:34.425308    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:10:34.425528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:34.425382    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:10:34.508340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:34.508249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:35.412462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:35.412429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:37.254482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:37.254441    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:38.099439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:38.099405    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:38.993571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:38.993533    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:40.929920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:40.929887    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:43.096638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.096542    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"]
Oct 30 01:10:43.097418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.097202    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy"
Oct 30 01:10:43.097418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.097215    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy"
Oct 30 01:10:43.097418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.097239    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy"
Oct 30 01:10:43.097418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.097244    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy"
Oct 30 01:10:43.097418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.097370    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy"
Oct 30 01:10:43.097418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.097382    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b19927d1-bfcb-4d1f-8a1d-79e9beed85f0" containerName="istio-proxy"
Oct 30 01:10:43.103607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.103583    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.120697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.120511    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"]
Oct 30 01:10:43.132011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.131981    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-975807-serving-cert\""
Oct 30 01:10:43.203062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.203023    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:43.231908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.231870    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.232068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.231926    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.232222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.232205    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7fd9\" (UniqueName: \"kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.232303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.232284    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.313958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.313920    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:43.362308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.338123    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7fd9\" (UniqueName: \"kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.362308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.338210    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.362308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.338306    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.362308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.338361    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.362308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.347079    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.389866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.389209    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.389866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.389652    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.426340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.414647    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7fd9\" (UniqueName: \"kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9\") pod \"maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.426340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.419281    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:43.731699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.731673    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"]
Oct 30 01:10:43.732262 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:43.732240    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b42540_ac44_4a16_95f6_611d39ade73e.slice/crio-16493be4d490ad7f42a05d7e2aa22d4dd253513170b28255d9df491555697ae1 WatchSource:0}: Error finding container 16493be4d490ad7f42a05d7e2aa22d4dd253513170b28255d9df491555697ae1: Status 404 returned error can't find the container with id 16493be4d490ad7f42a05d7e2aa22d4dd253513170b28255d9df491555697ae1
Oct 30 01:10:43.733721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.733702    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:10:43.963088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:43.963061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:44.440043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.439963    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"]
Oct 30 01:10:44.476419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.476388    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"]
Oct 30 01:10:44.575889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.575856    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:44.654375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.654336    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq" event={"ID":"d3b42540-ac44-4a16-95f6-611d39ade73e","Type":"ContainerStarted","Data":"a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be"}
Oct 30 01:10:44.654713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.654441    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq" event={"ID":"d3b42540-ac44-4a16-95f6-611d39ade73e","Type":"ContainerStarted","Data":"16493be4d490ad7f42a05d7e2aa22d4dd253513170b28255d9df491555697ae1"}
Oct 30 01:10:44.671342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.660165    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:44.701931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.698984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:44.735005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:44.734948    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq" podStartSLOduration=1.367058767 podStartE2EDuration="1.734932275s" totalImagesPullingTime="367.873508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:44.729553754 +0000 UTC m=+1752.758863985" watchObservedRunningTime="2026-10-30 01:10:44.734932275 +0000 UTC m=+1752.764242551"
Oct 30 01:10:45.085907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:45.084774    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:46.939749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:46.939709    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:48.728419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:48.728392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:49.441006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.440966    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh" podUID="94a99c88-90a1-4c56-ab07-64ed4e242094" containerName="payload-processing" containerID="cri-o://a4b3b9e0b61fcd9dc797c6754075be4f7c7ccafca8bce74adaf68052f8771797" gracePeriod=25
Oct 30 01:10:49.476776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.476742    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph" podUID="bb372fed-4f64-4844-bec2-df11368f47f9" containerName="payload-pre-processing" containerID="cri-o://5f291fa2fac05b43af98591911923a57e2056d6b73b8665ae1c9b0d8928a5566" gracePeriod=25
Oct 30 01:10:49.715448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.715398    4577 generic.go:356] "Generic (PLEG): container finished" podID="94a99c88-90a1-4c56-ab07-64ed4e242094" containerID="a4b3b9e0b61fcd9dc797c6754075be4f7c7ccafca8bce74adaf68052f8771797" exitCode=0
Oct 30 01:10:49.715676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.715650    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh" event={"ID":"94a99c88-90a1-4c56-ab07-64ed4e242094","Type":"ContainerDied","Data":"a4b3b9e0b61fcd9dc797c6754075be4f7c7ccafca8bce74adaf68052f8771797"}
Oct 30 01:10:49.723583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.722472    4577 generic.go:356] "Generic (PLEG): container finished" podID="bb372fed-4f64-4844-bec2-df11368f47f9" containerID="5f291fa2fac05b43af98591911923a57e2056d6b73b8665ae1c9b0d8928a5566" exitCode=0
Oct 30 01:10:49.724304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.723876    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph" event={"ID":"bb372fed-4f64-4844-bec2-df11368f47f9","Type":"ContainerDied","Data":"5f291fa2fac05b43af98591911923a57e2056d6b73b8665ae1c9b0d8928a5566"}
Oct 30 01:10:49.789598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.786190    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:10:49.817495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.817473    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:10:49.905140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.905100    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9\" (UniqueName: \"kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9\") pod \"94a99c88-90a1-4c56-ab07-64ed4e242094\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") "
Oct 30 01:10:49.905330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.905209    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q\" (UniqueName: \"kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q\") pod \"bb372fed-4f64-4844-bec2-df11368f47f9\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") "
Oct 30 01:10:49.905425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.905358    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume\") pod \"bb372fed-4f64-4844-bec2-df11368f47f9\" (UID: \"bb372fed-4f64-4844-bec2-df11368f47f9\") "
Oct 30 01:10:49.905492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.905449    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume\") pod \"94a99c88-90a1-4c56-ab07-64ed4e242094\" (UID: \"94a99c88-90a1-4c56-ab07-64ed4e242094\") "
Oct 30 01:10:49.906327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.906298    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume" pod "94a99c88-90a1-4c56-ab07-64ed4e242094" (UID: "94a99c88-90a1-4c56-ab07-64ed4e242094"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:49.907060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.907033    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume" pod "bb372fed-4f64-4844-bec2-df11368f47f9" (UID: "bb372fed-4f64-4844-bec2-df11368f47f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:49.909096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.909068    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9" pod "94a99c88-90a1-4c56-ab07-64ed4e242094" (UID: "94a99c88-90a1-4c56-ab07-64ed4e242094"). InnerVolumeSpecName "kube-api-access-rlhk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:49.909743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:49.909716    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q" pod "bb372fed-4f64-4844-bec2-df11368f47f9" (UID: "bb372fed-4f64-4844-bec2-df11368f47f9"). InnerVolumeSpecName "kube-api-access-ztn6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:50.007250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.007219    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlhk9\" (UniqueName: \"kubernetes.io/projected/94a99c88-90a1-4c56-ab07-64ed4e242094-kube-api-access-rlhk9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:50.007250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.007248    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztn6q\" (UniqueName: \"kubernetes.io/projected/bb372fed-4f64-4844-bec2-df11368f47f9-kube-api-access-ztn6q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:50.007468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.007258    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb372fed-4f64-4844-bec2-df11368f47f9-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:50.007468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.007268    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94a99c88-90a1-4c56-ab07-64ed4e242094-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:50.667665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.667637    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:10:50.735421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.735379    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph" event={"ID":"bb372fed-4f64-4844-bec2-df11368f47f9","Type":"ContainerDied","Data":"e32a815b2a04aff3fa9b1a1c39c2448b75e9d0fe03d9c6c844402b903f161c52"}
Oct 30 01:10:50.735610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.735461    4577 scope.go:122] "RemoveContainer" containerID="5f291fa2fac05b43af98591911923a57e2056d6b73b8665ae1c9b0d8928a5566"
Oct 30 01:10:50.735699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.735629    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"
Oct 30 01:10:50.740969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.740605    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh" event={"ID":"94a99c88-90a1-4c56-ab07-64ed4e242094","Type":"ContainerDied","Data":"991724afdb9ef6f8ad45d66ba76ef6c5887d8c59f5ee5aee5256123e02f99033"}
Oct 30 01:10:50.740969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.740702    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"
Oct 30 01:10:50.753079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.753058    4577 scope.go:122] "RemoveContainer" containerID="a4b3b9e0b61fcd9dc797c6754075be4f7c7ccafca8bce74adaf68052f8771797"
Oct 30 01:10:50.794336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.794305    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"]
Oct 30 01:10:50.800570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.800546    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-50d4ea-65fbffd984-s7nlh"]
Oct 30 01:10:50.840525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.840484    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"]
Oct 30 01:10:50.845953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.845929    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-50d4ea-f85f57b67-m9zph"]
Oct 30 01:10:50.999952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:50.999862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:52.112484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.112450    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94a99c88-90a1-4c56-ab07-64ed4e242094" path="/var/lib/kubelet/pods/94a99c88-90a1-4c56-ab07-64ed4e242094/volumes"
Oct 30 01:10:52.113068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.112881    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb372fed-4f64-4844-bec2-df11368f47f9" path="/var/lib/kubelet/pods/bb372fed-4f64-4844-bec2-df11368f47f9/volumes"
Oct 30 01:10:52.462033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.461940    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"]
Oct 30 01:10:52.462224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.462178    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj" podUID="9ee034c4-a2fd-43f1-84da-e10cf534d5b6" containerName="maas-api" containerID="cri-o://30e9694bf9ee4b751c3c72fdcd594c33fbb3b692627067d012203d38e158c796" gracePeriod=30
Oct 30 01:10:52.526547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.526510    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:52.791516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.791447    4577 generic.go:356] "Generic (PLEG): container finished" podID="9ee034c4-a2fd-43f1-84da-e10cf534d5b6" containerID="30e9694bf9ee4b751c3c72fdcd594c33fbb3b692627067d012203d38e158c796" exitCode=0
Oct 30 01:10:52.791987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.791961    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj" event={"ID":"9ee034c4-a2fd-43f1-84da-e10cf534d5b6","Type":"ContainerDied","Data":"30e9694bf9ee4b751c3c72fdcd594c33fbb3b692627067d012203d38e158c796"}
Oct 30 01:10:52.874133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.874106    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:10:52.974417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.970802    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m\") pod \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") "
Oct 30 01:10:52.974417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.970910    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs\") pod \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") "
Oct 30 01:10:52.974417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.970992    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca\") pod \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") "
Oct 30 01:10:52.974417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.971013    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls\") pod \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\" (UID: \"9ee034c4-a2fd-43f1-84da-e10cf534d5b6\") "
Oct 30 01:10:52.979142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.979090    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca" pod "9ee034c4-a2fd-43f1-84da-e10cf534d5b6" (UID: "9ee034c4-a2fd-43f1-84da-e10cf534d5b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:52.994803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.994771    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m" pod "9ee034c4-a2fd-43f1-84da-e10cf534d5b6" (UID: "9ee034c4-a2fd-43f1-84da-e10cf534d5b6"). InnerVolumeSpecName "kube-api-access-bcl6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:52.995010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.994769    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls" pod "9ee034c4-a2fd-43f1-84da-e10cf534d5b6" (UID: "9ee034c4-a2fd-43f1-84da-e10cf534d5b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:10:52.996588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:52.996547    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs" pod "9ee034c4-a2fd-43f1-84da-e10cf534d5b6" (UID: "9ee034c4-a2fd-43f1-84da-e10cf534d5b6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:10:53.048214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.048115    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:53.074148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.074119    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:53.074148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.074148    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:53.074339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.074159    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:53.074339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.074174    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcl6m\" (UniqueName: \"kubernetes.io/projected/9ee034c4-a2fd-43f1-84da-e10cf534d5b6-kube-api-access-bcl6m\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:10:53.078127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078102    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"]
Oct 30 01:10:53.078764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078749    4577 cpu_manager.go:414] "containerMap: removing container" podUID="bb372fed-4f64-4844-bec2-df11368f47f9" containerName="payload-pre-processing"
Oct 30 01:10:53.078802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078766    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb372fed-4f64-4844-bec2-df11368f47f9" containerName="payload-pre-processing"
Oct 30 01:10:53.078802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078800    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9ee034c4-a2fd-43f1-84da-e10cf534d5b6" containerName="maas-api"
Oct 30 01:10:53.078895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078805    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ee034c4-a2fd-43f1-84da-e10cf534d5b6" containerName="maas-api"
Oct 30 01:10:53.078895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078825    4577 cpu_manager.go:414] "containerMap: removing container" podUID="94a99c88-90a1-4c56-ab07-64ed4e242094" containerName="payload-processing"
Oct 30 01:10:53.078895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078830    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94a99c88-90a1-4c56-ab07-64ed4e242094" containerName="payload-processing"
Oct 30 01:10:53.078986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078933    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb372fed-4f64-4844-bec2-df11368f47f9" containerName="payload-pre-processing"
Oct 30 01:10:53.078986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078949    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ee034c4-a2fd-43f1-84da-e10cf534d5b6" containerName="maas-api"
Oct 30 01:10:53.078986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.078957    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="94a99c88-90a1-4c56-ab07-64ed4e242094" containerName="payload-processing"
Oct 30 01:10:53.082422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.082404    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.099114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.099089    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-5f8cbc-serving-cert\""
Oct 30 01:10:53.102067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.102042    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"]
Oct 30 01:10:53.162491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.162447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:53.175010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.174971    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.175146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.175084    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.175462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.175444    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.175558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.175479    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfxn8\" (UniqueName: \"kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.279870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.276995    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.279870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.277070    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfxn8\" (UniqueName: \"kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.279870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.277161    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.279870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.277238    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.293873 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:53.290963    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-serving-cert: secret "maas-api-e2e-praxis-a-5f8cbc-serving-cert" not found
Oct 30 01:10:53.293873 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:53.291087    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls podName:a383cf73-2c48-4143-9cc1-8f4f1bdfea1f nodeName:}" failed. No retries permitted until 2026-10-30 01:10:53.791063703 +0000 UTC m=+1761.820373930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls") pod "maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" (UID: "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f") : secret "maas-api-e2e-praxis-a-5f8cbc-serving-cert" not found
Oct 30 01:10:53.295145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.295117    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.295337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.295261    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.296877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.296102    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:53.317133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.317057    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfxn8\" (UniqueName: \"kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.447068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.447030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:53.666333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.666252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:53.800547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.800524    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"
Oct 30 01:10:53.800547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.800528    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj" event={"ID":"9ee034c4-a2fd-43f1-84da-e10cf534d5b6","Type":"ContainerDied","Data":"a6d8c5be07fba261f56a479a9ff5dbd341c323815e53630c3de7f13fa6447c8d"}
Oct 30 01:10:53.800806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.800608    4577 scope.go:122] "RemoveContainer" containerID="30e9694bf9ee4b751c3c72fdcd594c33fbb3b692627067d012203d38e158c796"
Oct 30 01:10:53.866168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.866132    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"]
Oct 30 01:10:53.879213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.879185    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-50d4ea-fcfff4b54-4czgj"]
Oct 30 01:10:53.885400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.885375    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.888157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.888137    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:53.999743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:53.999710    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:54.121145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:54.121109    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ee034c4-a2fd-43f1-84da-e10cf534d5b6" path="/var/lib/kubelet/pods/9ee034c4-a2fd-43f1-84da-e10cf534d5b6/volumes"
Oct 30 01:10:54.394412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:54.394193    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"]
Oct 30 01:10:54.394412 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:54.394379    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda383cf73_2c48_4143_9cc1_8f4f1bdfea1f.slice/crio-051816c6bb565d7b8d97e05c6bb561451de304b9aab9589953308fb17211c404 WatchSource:0}: Error finding container 051816c6bb565d7b8d97e05c6bb561451de304b9aab9589953308fb17211c404: Status 404 returned error can't find the container with id 051816c6bb565d7b8d97e05c6bb561451de304b9aab9589953308fb17211c404
Oct 30 01:10:54.810675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:54.810641    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" event={"ID":"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f","Type":"ContainerStarted","Data":"051816c6bb565d7b8d97e05c6bb561451de304b9aab9589953308fb17211c404"}
Oct 30 01:10:54.996342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:54.996266    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"]
Oct 30 01:10:55.000598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.000580    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.021401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.021377    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"]
Oct 30 01:10:55.024467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.024438    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-88a39d-serving-cert\""
Oct 30 01:10:55.093198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.093160    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:55.118454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.118420    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.118705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.118485    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.118705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.118525    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fh4x\" (UniqueName: \"kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.118705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.118668    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.216871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.216444    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.225825    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.225941    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.226007    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fh4x\" (UniqueName: \"kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.226103    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:55.227365    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-serving-cert: secret "maas-api-e2e-praxis-b-88a39d-serving-cert" not found
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:55.227428    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls podName:e171ba35-6cef-477e-a66a-090bc73715c9 nodeName:}" failed. No retries permitted until 2026-10-30 01:10:55.727405105 +0000 UTC m=+1763.756715328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls") pod "maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" (UID: "e171ba35-6cef-477e-a66a-090bc73715c9") : secret "maas-api-e2e-praxis-b-88a39d-serving-cert" not found
Oct 30 01:10:55.235096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.228578    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.254953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.246058    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.261449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.261409    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fh4x\" (UniqueName: \"kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.384205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.384178    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:55.734101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.734065    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.736633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.736613    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\") pod \"maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:55.821096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.821045    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" event={"ID":"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f","Type":"ContainerStarted","Data":"df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8"}
Oct 30 01:10:55.821096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.821129    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:10:55.870705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.870655    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" podStartSLOduration=2.522840929 podStartE2EDuration="2.870639607s" totalImagesPullingTime="347.798678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:55.867324856 +0000 UTC m=+1763.896635100" watchObservedRunningTime="2026-10-30 01:10:55.870639607 +0000 UTC m=+1763.899949864"
Oct 30 01:10:55.915545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:55.915389    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:56.297401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:56.296545    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"]
Oct 30 01:10:56.303327 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:10:56.303287    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode171ba35_6cef_477e_a66a_090bc73715c9.slice/crio-a6cba6491df6c98d3c630fb07097aab960cf88a127d541a6ae6ea5403b6c8705 WatchSource:0}: Error finding container a6cba6491df6c98d3c630fb07097aab960cf88a127d541a6ae6ea5403b6c8705: Status 404 returned error can't find the container with id a6cba6491df6c98d3c630fb07097aab960cf88a127d541a6ae6ea5403b6c8705
Oct 30 01:10:56.830041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:56.830003    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" event={"ID":"e171ba35-6cef-477e-a66a-090bc73715c9","Type":"ContainerStarted","Data":"82b715126cf0700500b18e4ea6bb46b47c506354a9c2da8fccc656f6b44ba0c8"}
Oct 30 01:10:56.830498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:56.830062    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" event={"ID":"e171ba35-6cef-477e-a66a-090bc73715c9","Type":"ContainerStarted","Data":"a6cba6491df6c98d3c630fb07097aab960cf88a127d541a6ae6ea5403b6c8705"}
Oct 30 01:10:56.830498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:56.830216    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:10:56.882304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:56.881354    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" podStartSLOduration=2.5887140730000002 podStartE2EDuration="2.881335664s" totalImagesPullingTime="292.621591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:10:56.874929316 +0000 UTC m=+1764.904239549" watchObservedRunningTime="2026-10-30 01:10:56.881335664 +0000 UTC m=+1764.910645896"
Oct 30 01:10:56.915318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:56.915287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:10:57.065288 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:10:57.065248    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:10:58.809704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:10:58.809668    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:00.031611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:00.031581    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:00.143224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:00.143194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:00.337464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:00.337430    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"]
Oct 30 01:11:00.337656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:00.337639    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" containerID="cri-o://bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8" gracePeriod=30
Oct 30 01:11:00.382785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:00.382755    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:00.613004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:00.612913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:01.838123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:01.838095    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:11:02.480106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:02.480072    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:02.845058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:02.844970    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:11:03.606784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:03.606744    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:11:03.607015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:03.606831    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:11:04.316765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:04.316727    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:05.642437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.642405    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:11:05.773370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.773328    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.777193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.777162    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:11:05.777409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.777191    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.777627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.777606    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.777939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.777918    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:05.778110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778092    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:05.778226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778179    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.778296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778274    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.778427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778411    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.778490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778454    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.778567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778551    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.778630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.778602    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket\") pod \"0dd9de74-415f-46de-849d-d846276afdb7\" (UID: \"0dd9de74-415f-46de-849d-d846276afdb7\") "
Oct 30 01:11:05.779467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.779189    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:05.779467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.779228    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:05.779467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.779363    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.779467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.779386    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.779467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.779402    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0dd9de74-415f-46de-849d-d846276afdb7-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.779467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.779415    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.783068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.783042    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:05.783878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.783807    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:05.784041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.784022    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "kube-api-access-p2ltn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:05.784344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.784323    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo" pod "0dd9de74-415f-46de-849d-d846276afdb7" (UID: "0dd9de74-415f-46de-849d-d846276afdb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:11:05.880400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.880320    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.880400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.880348    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2ltn\" (UniqueName: \"kubernetes.io/projected/0dd9de74-415f-46de-849d-d846276afdb7-kube-api-access-p2ltn\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.880400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.880357    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0dd9de74-415f-46de-849d-d846276afdb7-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.880400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.880367    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.880400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.880375    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0dd9de74-415f-46de-849d-d846276afdb7-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:05.921935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.921891    4577 generic.go:356] "Generic (PLEG): container finished" podID="0dd9de74-415f-46de-849d-d846276afdb7" containerID="bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8" exitCode=0
Oct 30 01:11:05.922138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.921992    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" event={"ID":"0dd9de74-415f-46de-849d-d846276afdb7","Type":"ContainerDied","Data":"bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8"}
Oct 30 01:11:05.922138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.922019    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"
Oct 30 01:11:05.922138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.922101    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn" event={"ID":"0dd9de74-415f-46de-849d-d846276afdb7","Type":"ContainerDied","Data":"2d9dcc6f27e56e9f9b2270fed6c27f2bd995b1a6434c32b82e20ba5de38162c3"}
Oct 30 01:11:05.922138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.922120    4577 scope.go:122] "RemoveContainer" containerID="bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8"
Oct 30 01:11:05.934076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.934054    4577 scope.go:122] "RemoveContainer" containerID="838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"
Oct 30 01:11:05.952874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.952067    4577 scope.go:122] "RemoveContainer" containerID="bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8"
Oct 30 01:11:05.952874 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:11:05.952699    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8\": container with ID starting with bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8 not found: ID does not exist" containerID="bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8"
Oct 30 01:11:05.952874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.952739    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8"} err="failed to get container status \"bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8\": rpc error: code = NotFound desc = could not find container \"bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8\": container with ID starting with bbeb42ca7f71b7f5ec691b4eb2992f224628fc3c2ff22f1af640e95fe00578e8 not found: ID does not exist"
Oct 30 01:11:05.952874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.952765    4577 scope.go:122] "RemoveContainer" containerID="838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"
Oct 30 01:11:05.953219 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:11:05.953064    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772\": container with ID starting with 838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772 not found: ID does not exist" containerID="838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"
Oct 30 01:11:05.953219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.953090    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772"} err="failed to get container status \"838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772\": rpc error: code = NotFound desc = could not find container \"838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772\": container with ID starting with 838d0c2baa62f980343b0b87091f1d0dd785e516505ce07852eabd8f45a89772 not found: ID does not exist"
Oct 30 01:11:05.990422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.990384    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"]
Oct 30 01:11:05.995269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:05.995244    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-50d4ea-openshift-default-7c4dc4b4c7-x96wn"]
Oct 30 01:11:06.113636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:06.113599    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dd9de74-415f-46de-849d-d846276afdb7" path="/var/lib/kubelet/pods/0dd9de74-415f-46de-849d-d846276afdb7/volumes"
Oct 30 01:11:06.154019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:06.153913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:08.460866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:08.460817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:10.241545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:10.241510    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:12.109085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:12.109052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:13.861252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:13.861215    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:14.996341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:14.996305    4577 generic.go:356] "Generic (PLEG): container finished" podID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerID="69a8635b3e117b242f4268f749c23e9cf2786c92850dcffa4bf0dee76e8b1525" exitCode=0
Oct 30 01:11:14.996821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:14.996378    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" event={"ID":"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820","Type":"ContainerDied","Data":"69a8635b3e117b242f4268f749c23e9cf2786c92850dcffa4bf0dee76e8b1525"}
Oct 30 01:11:14.996821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:14.996815    4577 scope.go:122] "RemoveContainer" containerID="69a8635b3e117b242f4268f749c23e9cf2786c92850dcffa4bf0dee76e8b1525"
Oct 30 01:11:14.998505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:14.998482    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:14.998562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:14.998550    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:14.998607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:14.998579    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:15.774997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:15.774947    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:16.006687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:16.006649    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" event={"ID":"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820","Type":"ContainerStarted","Data":"bb1f8866d84dc06750b7d33c50c74e88294103c4f16fe0ed5548b6af8a32394d"}
Oct 30 01:11:16.129331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:16.129238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:16.342174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:16.342137    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:11:16.343581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:16.343552    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused" start-of-body=
Oct 30 01:11:16.343717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:16.343604    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused"
Oct 30 01:11:17.341928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:17.341893    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused" start-of-body=
Oct 30 01:11:17.342387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:17.341950    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused"
Oct 30 01:11:18.106236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:18.106193    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:18.342401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:18.342361    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused" start-of-body=
Oct 30 01:11:18.342948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:18.342415    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused"
Oct 30 01:11:19.341891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:19.341825    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused" start-of-body=
Oct 30 01:11:19.342113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:19.341913    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused"
Oct 30 01:11:19.360075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:19.360018    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:11:20.012939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:20.012880    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:20.341971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:20.341885    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused" start-of-body=
Oct 30 01:11:20.341971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:20.341957    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.176:15021/healthz/ready\": dial tcp 10.134.0.176:15021: connect: connection refused"
Oct 30 01:11:21.364213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:21.364176    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:11:21.370385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:21.370347    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:11:21.541924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:21.541888    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:21.707438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:21.707333    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:21.890990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:21.890949    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:22.718007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:22.717977    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:22.843970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:22.843937    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:23.328708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:23.328678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:23.572415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:23.572326    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:23.909093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:23.909060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:25.844898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:25.844765    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:26.530677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:26.530622    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:26.651783 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:26.651741    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:27.020790 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:11:27.020738    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:11:27.507703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:27.507671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:27.793926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:27.791574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:29.322557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:29.322524    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:29.458935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:29.456984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:29.780043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:29.780011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:29.901871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:29.896094    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:30.141823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.141746    4577 generic.go:356] "Generic (PLEG): container finished" podID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerID="e3c34069a2e2ee83bc5c4ac3a446490c7ef0e921aea7f7001faf1b7fa50df31a" exitCode=0
Oct 30 01:11:30.142005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.141818    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" event={"ID":"e08d7191-2e5f-4812-b832-669b3aedaac8","Type":"ContainerDied","Data":"e3c34069a2e2ee83bc5c4ac3a446490c7ef0e921aea7f7001faf1b7fa50df31a"}
Oct 30 01:11:30.142280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.142263    4577 scope.go:122] "RemoveContainer" containerID="e3c34069a2e2ee83bc5c4ac3a446490c7ef0e921aea7f7001faf1b7fa50df31a"
Oct 30 01:11:30.144004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.143978    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:30.144077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.144048    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:30.144116 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.144080    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:30.536389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:30.536349    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:11:31.177388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:31.177345    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" event={"ID":"e08d7191-2e5f-4812-b832-669b3aedaac8","Type":"ContainerStarted","Data":"b2afa0087e8a12ea1ee0a7074adebd768f4f153623eca163c8caa51c33d06998"}
Oct 30 01:11:31.621869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:31.620326    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:11:31.621869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:31.620482    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 30 01:11:31.621869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:31.620525    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 30 01:11:31.655871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:31.655444    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:32.630991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:32.626980    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 30 01:11:32.630991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:32.627039    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 30 01:11:33.617923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:33.617875    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 30 01:11:33.618105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:33.617961    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 30 01:11:34.014491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:34.014447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:34.617955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:34.617914    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 30 01:11:34.618199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:34.618007    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 30 01:11:35.621961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:35.621923    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 30 01:11:35.622440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:35.622409    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 30 01:11:35.904579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:35.904479    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:36.618267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:36.618232    4577 patch_prober.go:29] interesting pod/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused" start-of-body=
Oct 30 01:11:36.618487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:36.618290    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.180:15021/healthz/ready\": dial tcp 10.134.0.180:15021: connect: connection refused"
Oct 30 01:11:37.622602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:37.622565    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:11:37.623099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:37.622945    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:11:37.623656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:37.623640    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:11:37.752582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:37.752547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:39.650305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:39.650272    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:40.918595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:40.918558    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:41.030780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:41.030745    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:42.012373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.012339    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:42.322238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.322203    4577 generic.go:356] "Generic (PLEG): container finished" podID="0464e965-6efa-49ae-a42d-900029d35dbc" containerID="99168b2a20b3f8d1db384c023678394e70b301cb65b85bf1d734a4d532fd0e10" exitCode=0
Oct 30 01:11:42.322429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.322275    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" event={"ID":"0464e965-6efa-49ae-a42d-900029d35dbc","Type":"ContainerDied","Data":"99168b2a20b3f8d1db384c023678394e70b301cb65b85bf1d734a4d532fd0e10"}
Oct 30 01:11:42.322641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.322628    4577 scope.go:122] "RemoveContainer" containerID="99168b2a20b3f8d1db384c023678394e70b301cb65b85bf1d734a4d532fd0e10"
Oct 30 01:11:42.324087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.324056    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:42.324223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.324140    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:42.324223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:42.324211    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:43.331693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:43.331660    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" event={"ID":"0464e965-6efa-49ae-a42d-900029d35dbc","Type":"ContainerStarted","Data":"b648baf58040354a8e6afc641b59fcbdbc1d42b9c4c03f40040145fd2911df61"}
Oct 30 01:11:43.811600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:43.811567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:44.021189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:44.021160    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:11:44.022489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:44.022464    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused" start-of-body=
Oct 30 01:11:44.022633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:44.022520    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused"
Oct 30 01:11:45.021599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:45.021569    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused" start-of-body=
Oct 30 01:11:45.022064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:45.021632    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused"
Oct 30 01:11:45.494285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:45.494235    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:45.537232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:45.537051    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:11:46.021241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:46.021204    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused" start-of-body=
Oct 30 01:11:46.021428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:46.021263    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused"
Oct 30 01:11:47.021512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:47.021478    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused" start-of-body=
Oct 30 01:11:47.021968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:47.021528    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused"
Oct 30 01:11:47.319071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:47.318984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:48.021183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:48.021147    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused" start-of-body=
Oct 30 01:11:48.021402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:48.021212    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused"
Oct 30 01:11:48.741940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:48.738958    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"]
Oct 30 01:11:48.741940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:48.739635    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96" podUID="1c5a124b-126f-425a-bbbe-92433b0981e3" containerName="maas-api" containerID="cri-o://affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb" gracePeriod=30
Oct 30 01:11:48.803020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:48.802487    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:49.090379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.084640    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:11:49.091806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.091717    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:11:49.301087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.300678    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:11:49.426921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.417345    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls\") pod \"1c5a124b-126f-425a-bbbe-92433b0981e3\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") "
Oct 30 01:11:49.426921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.417407    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs\" (UniqueName: \"kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs\") pod \"1c5a124b-126f-425a-bbbe-92433b0981e3\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") "
Oct 30 01:11:49.426921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.417574    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs\") pod \"1c5a124b-126f-425a-bbbe-92433b0981e3\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") "
Oct 30 01:11:49.426921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.417621    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca\") pod \"1c5a124b-126f-425a-bbbe-92433b0981e3\" (UID: \"1c5a124b-126f-425a-bbbe-92433b0981e3\") "
Oct 30 01:11:49.442868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.438087    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca" pod "1c5a124b-126f-425a-bbbe-92433b0981e3" (UID: "1c5a124b-126f-425a-bbbe-92433b0981e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:11:49.506883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.477370    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:49.506883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.499088    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs" pod "1c5a124b-126f-425a-bbbe-92433b0981e3" (UID: "1c5a124b-126f-425a-bbbe-92433b0981e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:11:49.506883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.499207    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs" pod "1c5a124b-126f-425a-bbbe-92433b0981e3" (UID: "1c5a124b-126f-425a-bbbe-92433b0981e3"). InnerVolumeSpecName "kube-api-access-chdqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:49.506883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.499256    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls" pod "1c5a124b-126f-425a-bbbe-92433b0981e3" (UID: "1c5a124b-126f-425a-bbbe-92433b0981e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:11:49.521560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.521531    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5a124b-126f-425a-bbbe-92433b0981e3-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:49.521782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.521762    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:49.521947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.521931    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chdqs\" (UniqueName: \"kubernetes.io/projected/1c5a124b-126f-425a-bbbe-92433b0981e3-kube-api-access-chdqs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:49.522096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.522081    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c5a124b-126f-425a-bbbe-92433b0981e3-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:11:49.523499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.523473    4577 generic.go:356] "Generic (PLEG): container finished" podID="1c5a124b-126f-425a-bbbe-92433b0981e3" containerID="affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb" exitCode=0
Oct 30 01:11:49.523694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.523525    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"
Oct 30 01:11:49.524712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.523542    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96" event={"ID":"1c5a124b-126f-425a-bbbe-92433b0981e3","Type":"ContainerDied","Data":"affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb"}
Oct 30 01:11:49.524907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.524888    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96" event={"ID":"1c5a124b-126f-425a-bbbe-92433b0981e3","Type":"ContainerDied","Data":"f17d0b6a4bd2c42a2583a9c4d05bde5030b0d6cefab1c01db404804408832d9d"}
Oct 30 01:11:49.525037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.525023    4577 scope.go:122] "RemoveContainer" containerID="affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb"
Oct 30 01:11:49.553683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.553571    4577 scope.go:122] "RemoveContainer" containerID="affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb"
Oct 30 01:11:49.554032 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:11:49.553972    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb\": container with ID starting with affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb not found: ID does not exist" containerID="affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb"
Oct 30 01:11:49.554032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.554000    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb"} err="failed to get container status \"affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb\": rpc error: code = NotFound desc = could not find container \"affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb\": container with ID starting with affa04e84ec37e1fd9ad61f01ed9a563d750680f39e4b2516fe15fd72502dddb not found: ID does not exist"
Oct 30 01:11:49.593267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.593234    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"]
Oct 30 01:11:49.597088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.597061    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4ec143-789b867c96-9ct96"]
Oct 30 01:11:49.599512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.599483    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:49.730385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.730223    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:49.852870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:49.852255    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:50.075969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:50.075445    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:50.159871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:50.156108    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c5a124b-126f-425a-bbbe-92433b0981e3" path="/var/lib/kubelet/pods/1c5a124b-126f-425a-bbbe-92433b0981e3/volumes"
Oct 30 01:11:51.614224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:51.614190    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:53.331543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:53.331511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:54.243644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.243611    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"]
Oct 30 01:11:54.244346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244330    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy"
Oct 30 01:11:54.244411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244348    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy"
Oct 30 01:11:54.244411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244372    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1c5a124b-126f-425a-bbbe-92433b0981e3" containerName="maas-api"
Oct 30 01:11:54.244411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244380    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c5a124b-126f-425a-bbbe-92433b0981e3" containerName="maas-api"
Oct 30 01:11:54.244411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244403    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy"
Oct 30 01:11:54.244411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244408    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy"
Oct 30 01:11:54.244610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244510    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy"
Oct 30 01:11:54.244610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.244523    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c5a124b-126f-425a-bbbe-92433b0981e3" containerName="maas-api"
Oct 30 01:11:54.248352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.248329    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.274301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.274274    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"]
Oct 30 01:11:54.327023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.326995    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327044    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327150    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327192    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327211    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327300    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327343    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327411    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98pqc\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.327478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.327456    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428387    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428453    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428524    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428571    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428599    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428650    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428708    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428741    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98pqc\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428763    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428896    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.428932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.428922    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.429427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.429033    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.429427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.429129    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.429427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.429341    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.431525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.431502    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.431655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.431632    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.453822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.453795    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.454191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.454168    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98pqc\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.565758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.565670    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:54.892338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.892307    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"]
Oct 30 01:11:54.892592 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:11:54.892562    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333870ba_945f_4e5c_949a_ca525c615630.slice/crio-1863e62a560b511236e13f0ee516ade64838236ad7521a4b6b94ca9bd7e7a258 WatchSource:0}: Error finding container 1863e62a560b511236e13f0ee516ade64838236ad7521a4b6b94ca9bd7e7a258: Status 404 returned error can't find the container with id 1863e62a560b511236e13f0ee516ade64838236ad7521a4b6b94ca9bd7e7a258
Oct 30 01:11:54.895665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.895630    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:54.895740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.895701    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:54.895786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:54.895733    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:11:55.124213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:55.124172    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:55.598156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:55.598121    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" event={"ID":"333870ba-945f-4e5c-949a-ca525c615630","Type":"ContainerStarted","Data":"2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0"}
Oct 30 01:11:55.598551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:55.598177    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" event={"ID":"333870ba-945f-4e5c-949a-ca525c615630","Type":"ContainerStarted","Data":"1863e62a560b511236e13f0ee516ade64838236ad7521a4b6b94ca9bd7e7a258"}
Oct 30 01:11:55.680871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:55.680002    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podStartSLOduration=1.679983107 podStartE2EDuration="1.679983107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:11:55.663374745 +0000 UTC m=+1823.692685005" watchObservedRunningTime="2026-10-30 01:11:55.679983107 +0000 UTC m=+1823.709293339"
Oct 30 01:11:56.358869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.357906    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:56.480868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.480473    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:56.566259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.566223    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:11:56.568860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.568811    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused" start-of-body=
Oct 30 01:11:56.569115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.569090    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused"
Oct 30 01:11:56.685173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.685058    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:56.986557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:56.986508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:11:57.016775 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:11:57.016731    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:11:57.106871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:57.106556    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm_25441484-890b-4e6c-94e0-b4dd99376b8c/payload-processing/0.log"
Oct 30 01:11:57.295327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:57.295240    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm_25441484-890b-4e6c-94e0-b4dd99376b8c/payload-processing/0.log"
Oct 30 01:11:57.571924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:57.571158    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused" start-of-body=
Oct 30 01:11:57.571924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:57.571222    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused"
Oct 30 01:11:58.568871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:58.568208    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused" start-of-body=
Oct 30 01:11:58.568871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:58.568285    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused"
Oct 30 01:11:59.568871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:59.566225    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused" start-of-body=
Oct 30 01:11:59.568871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:59.566289    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused"
Oct 30 01:11:59.600751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:11:59.600706    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:00.569442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:00.569405    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused" start-of-body=
Oct 30 01:12:00.570101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:00.570071    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused"
Oct 30 01:12:01.529707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:01.529679    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:01.566370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:01.566328    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused" start-of-body=
Oct 30 01:12:01.566571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:01.566389    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.185:15021/healthz/ready\": dial tcp 10.134.0.185:15021: connect: connection refused"
Oct 30 01:12:02.653537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:02.653495    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:12:02.654945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:02.654920    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:12:02.655933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:02.655906    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:12:03.346397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:03.346360    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:05.082643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:05.082608    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:07.294732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:07.294699    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:07.631296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:07.631207    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"]
Oct 30 01:12:07.665258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:07.665227    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"]
Oct 30 01:12:07.762183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:07.762150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:07.889217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:07.888402    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:09.126276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:09.126238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:10.812637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:10.812601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:12.612629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:12.612550    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:12.632429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:12.632394    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d" podUID="975c8a5a-790b-4560-8d0d-feaf1a7155be" containerName="payload-processing" containerID="cri-o://f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0" gracePeriod=25
Oct 30 01:12:12.666739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:12.666027    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr" podUID="78fea24a-068c-438a-a7b9-d1be036ce3ec" containerName="payload-pre-processing" containerID="cri-o://6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696" gracePeriod=25
Oct 30 01:12:12.968500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:12.968148    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:12:13.005936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.005912    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:12:13.105813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.105785    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume\") pod \"975c8a5a-790b-4560-8d0d-feaf1a7155be\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") "
Oct 30 01:12:13.106009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.105866    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9\" (UniqueName: \"kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9\") pod \"975c8a5a-790b-4560-8d0d-feaf1a7155be\" (UID: \"975c8a5a-790b-4560-8d0d-feaf1a7155be\") "
Oct 30 01:12:13.106009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.105895    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume\") pod \"78fea24a-068c-438a-a7b9-d1be036ce3ec\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") "
Oct 30 01:12:13.106093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.106008    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk\" (UniqueName: \"kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk\") pod \"78fea24a-068c-438a-a7b9-d1be036ce3ec\" (UID: \"78fea24a-068c-438a-a7b9-d1be036ce3ec\") "
Oct 30 01:12:13.107017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.106973    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume" pod "78fea24a-068c-438a-a7b9-d1be036ce3ec" (UID: "78fea24a-068c-438a-a7b9-d1be036ce3ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:13.116939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.107224    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume" pod "975c8a5a-790b-4560-8d0d-feaf1a7155be" (UID: "975c8a5a-790b-4560-8d0d-feaf1a7155be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:13.116939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.108803    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk" pod "78fea24a-068c-438a-a7b9-d1be036ce3ec" (UID: "78fea24a-068c-438a-a7b9-d1be036ce3ec"). InnerVolumeSpecName "kube-api-access-2fshk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:13.116939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.109818    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9" pod "975c8a5a-790b-4560-8d0d-feaf1a7155be" (UID: "975c8a5a-790b-4560-8d0d-feaf1a7155be"). InnerVolumeSpecName "kube-api-access-b97f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:13.241343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.213229    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975c8a5a-790b-4560-8d0d-feaf1a7155be-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.241343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.213271    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b97f9\" (UniqueName: \"kubernetes.io/projected/975c8a5a-790b-4560-8d0d-feaf1a7155be-kube-api-access-b97f9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.241343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.213290    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fea24a-068c-438a-a7b9-d1be036ce3ec-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.241343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.213303    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fshk\" (UniqueName: \"kubernetes.io/projected/78fea24a-068c-438a-a7b9-d1be036ce3ec-kube-api-access-2fshk\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.746601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.746528    4577 generic.go:356] "Generic (PLEG): container finished" podID="975c8a5a-790b-4560-8d0d-feaf1a7155be" containerID="f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0" exitCode=0
Oct 30 01:12:13.747472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.747439    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"
Oct 30 01:12:13.748087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.748060    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d" event={"ID":"975c8a5a-790b-4560-8d0d-feaf1a7155be","Type":"ContainerDied","Data":"f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0"}
Oct 30 01:12:13.748284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.748268    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d" event={"ID":"975c8a5a-790b-4560-8d0d-feaf1a7155be","Type":"ContainerDied","Data":"8dc83a4e466d548ed63754b5951b78584b908d5700a16d911f810a379ddec2b4"}
Oct 30 01:12:13.748418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.748395    4577 scope.go:122] "RemoveContainer" containerID="f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0"
Oct 30 01:12:13.750226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.750193    4577 generic.go:356] "Generic (PLEG): container finished" podID="78fea24a-068c-438a-a7b9-d1be036ce3ec" containerID="6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696" exitCode=0
Oct 30 01:12:13.750343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.750249    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr" event={"ID":"78fea24a-068c-438a-a7b9-d1be036ce3ec","Type":"ContainerDied","Data":"6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696"}
Oct 30 01:12:13.750343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.750289    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr" event={"ID":"78fea24a-068c-438a-a7b9-d1be036ce3ec","Type":"ContainerDied","Data":"8efacc4838dcf4d9490f909feaa9d8b8376f6ac907c2ffa06a9af8f70a9badbb"}
Oct 30 01:12:13.750471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.750362    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"
Oct 30 01:12:13.763005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.762980    4577 scope.go:122] "RemoveContainer" containerID="f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0"
Oct 30 01:12:13.763387 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:12:13.763358    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0\": container with ID starting with f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0 not found: ID does not exist" containerID="f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0"
Oct 30 01:12:13.763464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.763398    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0"} err="failed to get container status \"f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0\": rpc error: code = NotFound desc = could not find container \"f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0\": container with ID starting with f53ca2012c08c2fd451262881855fdbf278ba9b2c45e726f3dd2f1920391cdf0 not found: ID does not exist"
Oct 30 01:12:13.763464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.763422    4577 scope.go:122] "RemoveContainer" containerID="6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696"
Oct 30 01:12:13.775355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.775327    4577 scope.go:122] "RemoveContainer" containerID="6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696"
Oct 30 01:12:13.775734 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:12:13.775712    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696\": container with ID starting with 6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696 not found: ID does not exist" containerID="6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696"
Oct 30 01:12:13.775812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.775743    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696"} err="failed to get container status \"6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696\": rpc error: code = NotFound desc = could not find container \"6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696\": container with ID starting with 6dde5325bf66c8028ec8cdb85a9f5bad51e7608283496ba96d2c9d35983bb696 not found: ID does not exist"
Oct 30 01:12:13.807195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.807013    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"]
Oct 30 01:12:13.820972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.820755    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-975807-749f496979-zrfzr"]
Oct 30 01:12:13.853922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.852192    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"]
Oct 30 01:12:13.855967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:13.855936    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-975807-c7697cb8b-dpf5d"]
Oct 30 01:12:14.141910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:14.141825    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78fea24a-068c-438a-a7b9-d1be036ce3ec" path="/var/lib/kubelet/pods/78fea24a-068c-438a-a7b9-d1be036ce3ec/volumes"
Oct 30 01:12:14.142461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:14.142443    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="975c8a5a-790b-4560-8d0d-feaf1a7155be" path="/var/lib/kubelet/pods/975c8a5a-790b-4560-8d0d-feaf1a7155be/volumes"
Oct 30 01:12:14.401382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:14.401284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:16.312386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:16.312344    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:17.315819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:17.315787    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:18.161557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:18.161522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:20.024089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:20.024041    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:21.674727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:21.674691    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:21.830979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:21.830942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:23.789927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:23.789890    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"]
Oct 30 01:12:23.790449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:23.790081    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq" podUID="d3b42540-ac44-4a16-95f6-611d39ade73e" containerName="maas-api" containerID="cri-o://a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be" gracePeriod=30
Oct 30 01:12:23.856205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:23.856173    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:24.178935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.178903    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:12:24.359866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.350323    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls\") pod \"d3b42540-ac44-4a16-95f6-611d39ade73e\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") "
Oct 30 01:12:24.359866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.350372    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs\") pod \"d3b42540-ac44-4a16-95f6-611d39ade73e\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") "
Oct 30 01:12:24.359866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.350450    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca\") pod \"d3b42540-ac44-4a16-95f6-611d39ade73e\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") "
Oct 30 01:12:24.359866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.350681    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9\" (UniqueName: \"kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9\") pod \"d3b42540-ac44-4a16-95f6-611d39ade73e\" (UID: \"d3b42540-ac44-4a16-95f6-611d39ade73e\") "
Oct 30 01:12:24.370391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.370358    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca" pod "d3b42540-ac44-4a16-95f6-611d39ade73e" (UID: "d3b42540-ac44-4a16-95f6-611d39ade73e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:24.374128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.374097    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls" pod "d3b42540-ac44-4a16-95f6-611d39ade73e" (UID: "d3b42540-ac44-4a16-95f6-611d39ade73e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:24.384815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.384787    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9" pod "d3b42540-ac44-4a16-95f6-611d39ade73e" (UID: "d3b42540-ac44-4a16-95f6-611d39ade73e"). InnerVolumeSpecName "kube-api-access-d7fd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:24.386992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.385205    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs" pod "d3b42540-ac44-4a16-95f6-611d39ade73e" (UID: "d3b42540-ac44-4a16-95f6-611d39ade73e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:24.452351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.452263    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.452351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.452295    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3b42540-ac44-4a16-95f6-611d39ade73e-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.452351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.452309    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b42540-ac44-4a16-95f6-611d39ade73e-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.452351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.452322    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7fd9\" (UniqueName: \"kubernetes.io/projected/d3b42540-ac44-4a16-95f6-611d39ade73e-kube-api-access-d7fd9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.470555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.470516    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:24.591128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.591091    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:24.718446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.718358    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:24.879950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.879906    4577 generic.go:356] "Generic (PLEG): container finished" podID="d3b42540-ac44-4a16-95f6-611d39ade73e" containerID="a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be" exitCode=0
Oct 30 01:12:24.880407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.880031    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"
Oct 30 01:12:24.880407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.880045    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq" event={"ID":"d3b42540-ac44-4a16-95f6-611d39ade73e","Type":"ContainerDied","Data":"a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be"}
Oct 30 01:12:24.880407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.880149    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq" event={"ID":"d3b42540-ac44-4a16-95f6-611d39ade73e","Type":"ContainerDied","Data":"16493be4d490ad7f42a05d7e2aa22d4dd253513170b28255d9df491555697ae1"}
Oct 30 01:12:24.880407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.880178    4577 scope.go:122] "RemoveContainer" containerID="a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be"
Oct 30 01:12:24.891612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.891586    4577 scope.go:122] "RemoveContainer" containerID="a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be"
Oct 30 01:12:24.892018 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:12:24.891995    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be\": container with ID starting with a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be not found: ID does not exist" containerID="a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be"
Oct 30 01:12:24.892105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.892026    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be"} err="failed to get container status \"a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be\": rpc error: code = NotFound desc = could not find container \"a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be\": container with ID starting with a378cd52bf827edc5259eb6e454040e56078b51eae995cd369b3f86d186a66be not found: ID does not exist"
Oct 30 01:12:24.938184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.938152    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"]
Oct 30 01:12:24.951874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:24.951802    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-975807-845fffc9bf-55wxq"]
Oct 30 01:12:25.096459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:25.096431    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:26.112186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:26.112159    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3b42540-ac44-4a16-95f6-611d39ade73e" path="/var/lib/kubelet/pods/d3b42540-ac44-4a16-95f6-611d39ade73e/volumes"
Oct 30 01:12:26.549973 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:26.549936    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:27.014903 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:12:27.014869    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:12:28.353871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:28.353303    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:29.752957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:29.750547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:29.838871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:29.833215    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"]
Oct 30 01:12:29.838871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:29.833495    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" podUID="a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" containerName="maas-api" containerID="cri-o://df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8" gracePeriod=30
Oct 30 01:12:29.888553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:29.888280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:30.004693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.004611    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:30.182708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.182685    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:12:30.268380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.268295    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8\" (UniqueName: \"kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8\") pod \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") "
Oct 30 01:12:30.268687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.268664    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs\") pod \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") "
Oct 30 01:12:30.268832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.268816    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\") pod \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") "
Oct 30 01:12:30.269260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.269236    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca\") pod \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\" (UID: \"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f\") "
Oct 30 01:12:30.279244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.279211    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca" pod "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" (UID: "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:30.310130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.310077    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8" pod "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" (UID: "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f"). InnerVolumeSpecName "kube-api-access-nfxn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:30.314057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.314030    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs" pod "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" (UID: "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:30.319352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.319309    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls" pod "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" (UID: "a383cf73-2c48-4143-9cc1-8f4f1bdfea1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:30.388082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.375946    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:30.388082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.375986    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfxn8\" (UniqueName: \"kubernetes.io/projected/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-kube-api-access-nfxn8\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:30.388082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.376001    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:30.388082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.376015    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:30.627019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.626908    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:30.932308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.932214    4577 generic.go:356] "Generic (PLEG): container finished" podID="a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" containerID="df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8" exitCode=0
Oct 30 01:12:30.932308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.932270    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"
Oct 30 01:12:30.932308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.932296    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" event={"ID":"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f","Type":"ContainerDied","Data":"df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8"}
Oct 30 01:12:30.932939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.932347    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz" event={"ID":"a383cf73-2c48-4143-9cc1-8f4f1bdfea1f","Type":"ContainerDied","Data":"051816c6bb565d7b8d97e05c6bb561451de304b9aab9589953308fb17211c404"}
Oct 30 01:12:30.932939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.932371    4577 scope.go:122] "RemoveContainer" containerID="df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8"
Oct 30 01:12:30.943392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.943370    4577 scope.go:122] "RemoveContainer" containerID="df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8"
Oct 30 01:12:30.943691 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:12:30.943667    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8\": container with ID starting with df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8 not found: ID does not exist" containerID="df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8"
Oct 30 01:12:30.943766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.943700    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8"} err="failed to get container status \"df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8\": rpc error: code = NotFound desc = could not find container \"df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8\": container with ID starting with df356b402e47d67fb89cc739d7a50069190ddabe93a39708236ca9c77f5bf9e8 not found: ID does not exist"
Oct 30 01:12:30.990398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:30.990362    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"]
Oct 30 01:12:31.004613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.004581    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-5f8cbc-86cb9b85dc-qs6sz"]
Oct 30 01:12:31.047431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.047401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:31.158208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.158177    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:31.382127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.382092    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:31.497614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.497579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:31.682459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.682379    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:31.707685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.707650    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"]
Oct 30 01:12:31.708162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.708132    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" containerID="cri-o://bb1f8866d84dc06750b7d33c50c74e88294103c4f16fe0ed5548b6af8a32394d" gracePeriod=30
Oct 30 01:12:31.798516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:31.798481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:32.111547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:32.111513    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" path="/var/lib/kubelet/pods/a383cf73-2c48-4143-9cc1-8f4f1bdfea1f/volumes"
Oct 30 01:12:32.540561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:32.540530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:34.446422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:34.446386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:36.195609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:36.195562    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:36.368806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:36.368760    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:12:36.369001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:36.368821    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:12:37.012999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.012966    4577 generic.go:356] "Generic (PLEG): container finished" podID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerID="bb1f8866d84dc06750b7d33c50c74e88294103c4f16fe0ed5548b6af8a32394d" exitCode=0
Oct 30 01:12:37.013144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.013037    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" event={"ID":"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820","Type":"ContainerDied","Data":"bb1f8866d84dc06750b7d33c50c74e88294103c4f16fe0ed5548b6af8a32394d"}
Oct 30 01:12:37.013144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.013099    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs" event={"ID":"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820","Type":"ContainerDied","Data":"39dc018039c074aa96fd22be4815ea556167e944ed7ca83bd6708268fbea14bc"}
Oct 30 01:12:37.013144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.013117    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39dc018039c074aa96fd22be4815ea556167e944ed7ca83bd6708268fbea14bc"
Oct 30 01:12:37.013267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.013166    4577 scope.go:122] "RemoveContainer" containerID="69a8635b3e117b242f4268f749c23e9cf2786c92850dcffa4bf0dee76e8b1525"
Oct 30 01:12:37.014342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.014243    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:12:37.115389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115355    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115420    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115456    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115588    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115639    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115671    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115758    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115917    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.115987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.115970    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs\") pod \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\" (UID: \"1ee84bb2-ddc3-49b9-a772-ffeef3ffc820\") "
Oct 30 01:12:37.116311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116278    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:37.116311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116306    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:37.116522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116468    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:37.116591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116538    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:37.116591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116549    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:37.116814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116794    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.117007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116819    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.117007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116833    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.117007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116882    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.117007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.116898    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.119219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.119190    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:12:37.119339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.119229    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:37.120680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.120647    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "kube-api-access-rhstw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:37.121022 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.121000    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy" pod "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" (UID: "1ee84bb2-ddc3-49b9-a772-ffeef3ffc820"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:37.218036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.217907    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.218036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.217943    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.218036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.217979    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhstw\" (UniqueName: \"kubernetes.io/projected/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-kube-api-access-rhstw\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.218036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.218006    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:37.906306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:37.906269    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:38.021191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.021162    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"
Oct 30 01:12:38.162304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.162210    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"]
Oct 30 01:12:38.167398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.167367    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-975807-openshift-default-6fd7f6dd9-nxjgs"]
Oct 30 01:12:38.612188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.612155    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:38.723199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.723163    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:38.906772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.906674    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"]
Oct 30 01:12:38.906986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.906943    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy" containerID="cri-o://b2afa0087e8a12ea1ee0a7074adebd768f4f153623eca163c8caa51c33d06998" gracePeriod=30
Oct 30 01:12:38.963050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:38.963011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:39.404410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.404378    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"]
Oct 30 01:12:39.404979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.404965    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d3b42540-ac44-4a16-95f6-611d39ade73e" containerName="maas-api"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.404981    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3b42540-ac44-4a16-95f6-611d39ade73e" containerName="maas-api"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.404997    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405002    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405009    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405013    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405031    4577 cpu_manager.go:414] "containerMap: removing container" podUID="78fea24a-068c-438a-a7b9-d1be036ce3ec" containerName="payload-pre-processing"
Oct 30 01:12:39.405035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405036    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78fea24a-068c-438a-a7b9-d1be036ce3ec" containerName="payload-pre-processing"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405050    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" containerName="maas-api"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405054    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" containerName="maas-api"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405062    4577 cpu_manager.go:414] "containerMap: removing container" podUID="975c8a5a-790b-4560-8d0d-feaf1a7155be" containerName="payload-processing"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405067    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="975c8a5a-790b-4560-8d0d-feaf1a7155be" containerName="payload-processing"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405149    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="975c8a5a-790b-4560-8d0d-feaf1a7155be" containerName="payload-processing"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405162    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3b42540-ac44-4a16-95f6-611d39ade73e" containerName="maas-api"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405170    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405176    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" containerName="istio-proxy"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405185    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="78fea24a-068c-438a-a7b9-d1be036ce3ec" containerName="payload-pre-processing"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405192    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dd9de74-415f-46de-849d-d846276afdb7" containerName="istio-proxy"
Oct 30 01:12:39.405258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.405198    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a383cf73-2c48-4143-9cc1-8f4f1bdfea1f" containerName="maas-api"
Oct 30 01:12:39.410277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.410258    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.431681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.431650    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"]
Oct 30 01:12:39.437182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.437156    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.443531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.443505    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"]
Oct 30 01:12:39.451475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451431    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.451741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451717    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.451905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451793    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.451905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451826    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.452061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451914    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.452061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451960    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.452061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.451994    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.452224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.452107    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.452224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.452152    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27x9j\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.471783 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.471746    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"]
Oct 30 01:12:39.553887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.553826    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.553924    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.553954    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.553982    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27x9j\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554025    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554074    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554144    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554205    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554287    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554317    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554360    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554399    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554411    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554490    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx8fr\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554519    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554563    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554595    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554610    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.554651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554624    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.555117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554763    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.555117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554884    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.555117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.554995    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.555655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.555631    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.557173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.557147    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.557463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.557442    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.579696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.579666    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.579832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.579807    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27x9j\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.656335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656230    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656325    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656388    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656430    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656464    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx8fr\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656515    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656592    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656633    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.656924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.656671    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.657874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.657754    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.657874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.657792    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.658994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.658283    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.658994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.658338    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.658994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.658952    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.660460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.660436    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.661332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.661297    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.681613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.681584    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.683016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.682995    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx8fr\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr\") pod \"e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") " pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:39.728313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.728278    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:39.757460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:39.757434    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:40.033612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.033574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:40.051939 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:12:40.051910    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba32f0d_75b2_4d77_90c6_be3def381eab.slice/crio-a1f31915cab2cb8b3086c327e979921d63b136c87628532d99e68467bc219925 WatchSource:0}: Error finding container a1f31915cab2cb8b3086c327e979921d63b136c87628532d99e68467bc219925: Status 404 returned error can't find the container with id a1f31915cab2cb8b3086c327e979921d63b136c87628532d99e68467bc219925
Oct 30 01:12:40.052899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.052869    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"]
Oct 30 01:12:40.054690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.054653    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:12:40.054760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.054739    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:12:40.054803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.054782    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:12:40.067331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.067296    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" event={"ID":"eba32f0d-75b2-4d77-90c6-be3def381eab","Type":"ContainerStarted","Data":"a1f31915cab2cb8b3086c327e979921d63b136c87628532d99e68467bc219925"}
Oct 30 01:12:40.098470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.098420    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"]
Oct 30 01:12:40.147813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.147052    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:12:40.147813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.147146    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:12:40.147813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.147190    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:12:40.153703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:40.153674    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ee84bb2-ddc3-49b9-a772-ffeef3ffc820" path="/var/lib/kubelet/pods/1ee84bb2-ddc3-49b9-a772-ffeef3ffc820/volumes"
Oct 30 01:12:41.113059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.113023    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" event={"ID":"a5d9c977-a868-48ad-86df-25432459b386","Type":"ContainerStarted","Data":"273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786"}
Oct 30 01:12:41.113581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.113122    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" event={"ID":"a5d9c977-a868-48ad-86df-25432459b386","Type":"ContainerStarted","Data":"c024ca445d81db96837bc4a88d8204353ea14392c7fed19709a8df3e47283523"}
Oct 30 01:12:41.115566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.115543    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" event={"ID":"eba32f0d-75b2-4d77-90c6-be3def381eab","Type":"ContainerStarted","Data":"c36112aee151ae3817ea669d8fef7c5b15a31e45671ccff7b62b0267a6498a65"}
Oct 30 01:12:41.193871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.193372    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podStartSLOduration=2.193352649 podStartE2EDuration="2.193352649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:12:41.178350983 +0000 UTC m=+1869.207661215" watchObservedRunningTime="2026-10-30 01:12:41.193352649 +0000 UTC m=+1869.222662880"
Oct 30 01:12:41.239089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.239031    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podStartSLOduration=2.239015854 podStartE2EDuration="2.239015854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:12:41.237890149 +0000 UTC m=+1869.267200380" watchObservedRunningTime="2026-10-30 01:12:41.239015854 +0000 UTC m=+1869.268326085"
Oct 30 01:12:41.729325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.729283    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:41.731537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.731509    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 30 01:12:41.731744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.731564    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 30 01:12:41.765870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.764137    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:41.765870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.764265    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused" start-of-body=
Oct 30 01:12:41.765870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:41.764309    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused"
Oct 30 01:12:42.159760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:42.159663    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:42.729211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:42.729183    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 30 01:12:42.729449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:42.729425    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 30 01:12:42.758776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:42.758743    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused" start-of-body=
Oct 30 01:12:42.759118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:42.759073    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused"
Oct 30 01:12:43.728785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:43.728753    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 30 01:12:43.729252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:43.728805    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 30 01:12:43.765870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:43.759576    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused" start-of-body=
Oct 30 01:12:43.765870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:43.759652    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused"
Oct 30 01:12:44.138325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.138293    4577 generic.go:356] "Generic (PLEG): container finished" podID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerID="b2afa0087e8a12ea1ee0a7074adebd768f4f153623eca163c8caa51c33d06998" exitCode=0
Oct 30 01:12:44.138502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.138375    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" event={"ID":"e08d7191-2e5f-4812-b832-669b3aedaac8","Type":"ContainerDied","Data":"b2afa0087e8a12ea1ee0a7074adebd768f4f153623eca163c8caa51c33d06998"}
Oct 30 01:12:44.138502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.138449    4577 scope.go:122] "RemoveContainer" containerID="e3c34069a2e2ee83bc5c4ac3a446490c7ef0e921aea7f7001faf1b7fa50df31a"
Oct 30 01:12:44.417059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.417032    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.499696    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.499745    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.499778    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.499831    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.499902    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.500032    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.500088    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.500197    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.501870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.500323    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data\") pod \"e08d7191-2e5f-4812-b832-669b3aedaac8\" (UID: \"e08d7191-2e5f-4812-b832-669b3aedaac8\") "
Oct 30 01:12:44.507065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.502761    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:44.507065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.502970    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:44.507065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.507018    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:44.508448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.507620    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:44.512861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.510937    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:44.526711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.522920    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:12:44.526711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.523103    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:44.526711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.524702    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:44.526711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.526477    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h" pod "e08d7191-2e5f-4812-b832-669b3aedaac8" (UID: "e08d7191-2e5f-4812-b832-669b3aedaac8"). InnerVolumeSpecName "kube-api-access-r722h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621051    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621075    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621085    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e08d7191-2e5f-4812-b832-669b3aedaac8-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621093    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621115    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621127    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621142    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08d7191-2e5f-4812-b832-669b3aedaac8-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621154    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.621630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.621167    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r722h\" (UniqueName: \"kubernetes.io/projected/e08d7191-2e5f-4812-b832-669b3aedaac8-kube-api-access-r722h\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:12:44.729123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.729082    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 30 01:12:44.729633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.729156    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 30 01:12:44.766874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.766353    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused" start-of-body=
Oct 30 01:12:44.766874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.766413    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused"
Oct 30 01:12:44.890926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:44.889900    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:45.161928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.155575    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf" event={"ID":"e08d7191-2e5f-4812-b832-669b3aedaac8","Type":"ContainerDied","Data":"c0e4bb130ad14e1f7f9ef96ad658b847c3e7db63d879046af105d1a6a8a13988"}
Oct 30 01:12:45.161928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.155652    4577 scope.go:122] "RemoveContainer" containerID="b2afa0087e8a12ea1ee0a7074adebd768f4f153623eca163c8caa51c33d06998"
Oct 30 01:12:45.161928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.155778    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"
Oct 30 01:12:45.302870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.298894    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"]
Oct 30 01:12:45.308911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.307285    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-5f8cbc-openshift-default-57fdc694df-vblzf"]
Oct 30 01:12:45.729107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.728648    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 30 01:12:45.729107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.728721    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 30 01:12:45.758322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.758283    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused" start-of-body=
Oct 30 01:12:45.758631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:45.758602    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused"
Oct 30 01:12:46.118497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:46.118390    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" path="/var/lib/kubelet/pods/e08d7191-2e5f-4812-b832-669b3aedaac8/volumes"
Oct 30 01:12:46.728764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:46.728728    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused" start-of-body=
Oct 30 01:12:46.729020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:46.728804    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": dial tcp 10.134.0.186:15021: connect: connection refused"
Oct 30 01:12:46.759231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:46.759193    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused" start-of-body=
Oct 30 01:12:46.759759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:46.759268    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": dial tcp 10.134.0.187:15021: connect: connection refused"
Oct 30 01:12:46.904309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:46.904274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:48.748751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:48.748715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:48.766015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:48.765978    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:12:48.766393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:48.766350    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:12:48.816429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:48.815986    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:12:48.816429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:48.816049    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:12:49.170382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.141478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:49.260738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.260703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:49.762448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.762408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:49.789230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.789171    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:12:49.789576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.789519    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:12:49.851032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.850989    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:12:49.851303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.851269    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:12:50.017919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:49.994789    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:50.276945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.276511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:50.531358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.531269    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:50.713873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.706451    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:50.823896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.823765    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:50.882335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.864616    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:12:50.882335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.881646    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:12:50.950872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.911942    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:12:50.950872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:50.931938    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:12:51.745833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:51.745799    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:51.747487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:51.747463    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:51.748158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:51.748139    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:12:51.750858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:51.750819    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:51.751653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:51.751633    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:51.752191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:51.752174    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:12:52.766579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:52.766545    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:54.531653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:54.531612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:55.485780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:55.485742    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"]
Oct 30 01:12:55.558563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:55.558526    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:55.996999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:55.996967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:56.601278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:56.601240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:56.712421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:56.712391    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:56.836878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:56.836805    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:57.014199 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:12:57.014160    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:12:57.219020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:57.218977    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:12:58.749670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:12:58.749571    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:00.544931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:00.544833    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:02.536638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:02.536596    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"]
Oct 30 01:13:02.537176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:02.536890    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:02.567259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:02.567220    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"]
Oct 30 01:13:02.669229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:02.669190    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:02.787838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:02.787749    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:03.863598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:03.863564    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:03.977509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:03.977473    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:04.346713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:04.346681    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:05.036119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:05.036087    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:06.902551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:06.902518    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:07.536979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:07.536940    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerName="payload-processing" containerID="cri-o://e7e72ac6fad90199fc94b88c9180140f0782ecee80028d00f8c7583f7babe5c8" gracePeriod=25
Oct 30 01:13:07.568308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:07.568252    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerName="payload-pre-processing" containerID="cri-o://db3ebf0a5b4239725fbb9f01abf39e472dea9503a717eb68337f49f8066d0573" gracePeriod=25
Oct 30 01:13:08.728299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:08.728260    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:09.483754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:09.483717    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:13:09.483989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:09.483807    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:13:10.499945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.499915    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:10.522007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.521943    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" podUID="2e575dc9-198f-4329-b2b0-54ade58440fa" containerName="main" containerID="cri-o://11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0" gracePeriod=45
Oct 30 01:13:10.537100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.536731    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" podUID="2e575dc9-198f-4329-b2b0-54ade58440fa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.177:8000/ready\": dial tcp 10.134.0.177:8000: connect: connection refused"
Oct 30 01:13:10.807001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.806974    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.843255    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9\" (UniqueName: \"kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9\") pod \"2e575dc9-198f-4329-b2b0-54ade58440fa\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") "
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.843376    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm\") pod \"2e575dc9-198f-4329-b2b0-54ade58440fa\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") "
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.843430    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir\") pod \"2e575dc9-198f-4329-b2b0-54ade58440fa\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") "
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.843495    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache\") pod \"2e575dc9-198f-4329-b2b0-54ade58440fa\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") "
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.843691    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs\" (UniqueName: \"kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs\") pod \"2e575dc9-198f-4329-b2b0-54ade58440fa\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") "
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.843770    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home\") pod \"2e575dc9-198f-4329-b2b0-54ade58440fa\" (UID: \"2e575dc9-198f-4329-b2b0-54ade58440fa\") "
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.862076    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache" pod "2e575dc9-198f-4329-b2b0-54ade58440fa" (UID: "2e575dc9-198f-4329-b2b0-54ade58440fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:13:10.862176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.862149    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir" pod "2e575dc9-198f-4329-b2b0-54ade58440fa" (UID: "2e575dc9-198f-4329-b2b0-54ade58440fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:13:10.863079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.863031    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home" pod "2e575dc9-198f-4329-b2b0-54ade58440fa" (UID: "2e575dc9-198f-4329-b2b0-54ade58440fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:13:10.935323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.935255    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm" pod "2e575dc9-198f-4329-b2b0-54ade58440fa" (UID: "2e575dc9-198f-4329-b2b0-54ade58440fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:13:10.936413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.936371    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9" pod "2e575dc9-198f-4329-b2b0-54ade58440fa" (UID: "2e575dc9-198f-4329-b2b0-54ade58440fa"). InnerVolumeSpecName "kube-api-access-x8vw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.951336    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.951367    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8vw9\" (UniqueName: \"kubernetes.io/projected/2e575dc9-198f-4329-b2b0-54ade58440fa-kube-api-access-x8vw9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.951381    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.951416    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.951429    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e575dc9-198f-4329-b2b0-54ade58440fa-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.955062    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs" pod "2e575dc9-198f-4329-b2b0-54ade58440fa" (UID: "2e575dc9-198f-4329-b2b0-54ade58440fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.959334    4577 generic.go:356] "Generic (PLEG): container finished" podID="2e575dc9-198f-4329-b2b0-54ade58440fa" containerID="11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0" exitCode=0
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.959473    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" event={"ID":"2e575dc9-198f-4329-b2b0-54ade58440fa","Type":"ContainerDied","Data":"11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0"}
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.959508    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w" event={"ID":"2e575dc9-198f-4329-b2b0-54ade58440fa","Type":"ContainerDied","Data":"1c67b2aabc4231d96ff38f80293a8138104514e297581c081f927f9894ed5e21"}
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.959533    4577 scope.go:122] "RemoveContainer" containerID="11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0"
Oct 30 01:13:10.962875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.959699    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"
Oct 30 01:13:10.995837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.995814    4577 scope.go:122] "RemoveContainer" containerID="11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0"
Oct 30 01:13:10.997080 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:13:10.997054    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0\": container with ID starting with 11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0 not found: ID does not exist" containerID="11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0"
Oct 30 01:13:10.997198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:10.997093    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0"} err="failed to get container status \"11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0\": rpc error: code = NotFound desc = could not find container \"11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0\": container with ID starting with 11cd05a462235df8877ae1baf768055eda2be068070a42a8ea632dfcaba5fcf0 not found: ID does not exist"
Oct 30 01:13:11.051192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:11.051108    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"]
Oct 30 01:13:11.053118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:11.053083    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e575dc9-198f-4329-b2b0-54ade58440fa-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:11.054791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:11.054767    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-f05249/ipp-route-f05249-kserve-74fdc7f7-7gl8w"]
Oct 30 01:13:12.115066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:12.115036    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e575dc9-198f-4329-b2b0-54ade58440fa" path="/var/lib/kubelet/pods/2e575dc9-198f-4329-b2b0-54ade58440fa/volumes"
Oct 30 01:13:12.406753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:12.406617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.537636    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"]
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.538646    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy"
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.538669    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy"
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.538697    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2e575dc9-198f-4329-b2b0-54ade58440fa" containerName="main"
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.538705    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e575dc9-198f-4329-b2b0-54ade58440fa" containerName="main"
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.538744    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy"
Oct 30 01:13:13.540865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.538752    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy"
Oct 30 01:13:13.548127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.548099    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy"
Oct 30 01:13:13.548357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.548340    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e575dc9-198f-4329-b2b0-54ade58440fa" containerName="main"
Oct 30 01:13:13.563930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.563206    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.575263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.568901    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"]
Oct 30 01:13:13.575263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.570058    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e08d7191-2e5f-4812-b832-669b3aedaac8" containerName="istio-proxy"
Oct 30 01:13:13.592332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.585639    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.643607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.643572    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"]
Oct 30 01:13:13.664966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.664933    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.665231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.665206    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.665380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.665360    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.665545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.665526    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.665705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.665688    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.665937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.665906    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.666128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.666110    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.666253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.666237    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.666398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.666383    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.666810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.666786    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"]
Oct 30 01:13:13.768118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768082    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.768329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768168    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.768329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768201    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.768329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768279    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.768555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768328    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.768555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768393    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.768555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768428    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.768728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768623    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.768728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768651    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.768728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768694    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.768929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768760    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.768929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768809    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.769058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768941    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.768974    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.769058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769035    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769064    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769084    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769110    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769176    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86jmg\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.769284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769238    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.769284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769245    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769623    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.769948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.769814    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.772010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.771987    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.772465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.772442    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.793684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.793615    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.794098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.794079    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.870460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870431    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.870692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870480    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.870692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870506    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.870692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870528    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.870692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870562    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.870692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870623    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870701    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86jmg\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870753    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870815    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870914    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870942    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.870970    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.871182    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.871709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.871496    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.873261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.873239    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.873456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.873440    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.895924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.895894    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.896174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.896154    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86jmg\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:13.928926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.928896    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:13.945349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:13.945314    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:14.271952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.271919    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"]
Oct 30 01:13:14.273105 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:13:14.273075    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48448591_cd0a_43a4_aacd_7aa72c32fdb0.slice/crio-51af48237d1aa3cbe23b380246ad07da023537d72aadfd49283d9d558b0d3d69 WatchSource:0}: Error finding container 51af48237d1aa3cbe23b380246ad07da023537d72aadfd49283d9d558b0d3d69: Status 404 returned error can't find the container with id 51af48237d1aa3cbe23b380246ad07da023537d72aadfd49283d9d558b0d3d69
Oct 30 01:13:14.275474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.275452    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"]
Oct 30 01:13:14.275749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.275719    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:14.275810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.275780    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:14.275882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.275810    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:14.286549 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:13:14.286521    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a34547_fc81_4348_9ce8_a7aaafa6723f.slice/crio-47e254383ccca04ca373cc612419c4f42eac3c83878bb9fa25e333e0fa9d7ee4 WatchSource:0}: Error finding container 47e254383ccca04ca373cc612419c4f42eac3c83878bb9fa25e333e0fa9d7ee4: Status 404 returned error can't find the container with id 47e254383ccca04ca373cc612419c4f42eac3c83878bb9fa25e333e0fa9d7ee4
Oct 30 01:13:14.288991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.288955    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:14.289103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.289047    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:14.289103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.289076    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:14.313004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:14.312972    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:15.119467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.118409    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" event={"ID":"48448591-cd0a-43a4-aacd-7aa72c32fdb0","Type":"ContainerStarted","Data":"672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d"}
Oct 30 01:13:15.119907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.119494    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" event={"ID":"48448591-cd0a-43a4-aacd-7aa72c32fdb0","Type":"ContainerStarted","Data":"51af48237d1aa3cbe23b380246ad07da023537d72aadfd49283d9d558b0d3d69"}
Oct 30 01:13:15.120983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.120962    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" event={"ID":"b6a34547-fc81-4348-9ce8-a7aaafa6723f","Type":"ContainerStarted","Data":"e641807a5d355611a8996147882b985ca1d680e22eee5c283f987db22a88525a"}
Oct 30 01:13:15.121093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.121000    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" event={"ID":"b6a34547-fc81-4348-9ce8-a7aaafa6723f","Type":"ContainerStarted","Data":"47e254383ccca04ca373cc612419c4f42eac3c83878bb9fa25e333e0fa9d7ee4"}
Oct 30 01:13:15.216544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.216488    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podStartSLOduration=2.2164715680000002 podStartE2EDuration="2.216471568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:15.20036587 +0000 UTC m=+1903.229676139" watchObservedRunningTime="2026-10-30 01:13:15.216471568 +0000 UTC m=+1903.245781799"
Oct 30 01:13:15.929311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.929277    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:15.930581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.930551    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused" start-of-body=
Oct 30 01:13:15.930947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.930920    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused"
Oct 30 01:13:15.948871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.947489    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:15.948871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.947617    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 01:13:15.948871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:15.947656    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 01:13:16.211935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:16.209327    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:16.929490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:16.929447    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused" start-of-body=
Oct 30 01:13:16.929680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:16.929515    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused"
Oct 30 01:13:16.969871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:16.967939    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 01:13:16.969871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:16.967999    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 01:13:17.929278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:17.929239    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused" start-of-body=
Oct 30 01:13:17.929775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:17.929316    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused"
Oct 30 01:13:17.959890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:17.950182    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 01:13:17.959890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:17.950258    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 01:13:18.144489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:18.144457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:18.929914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:18.929793    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused" start-of-body=
Oct 30 01:13:18.929914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:18.929883    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused"
Oct 30 01:13:18.952871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:18.950224    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 01:13:18.952871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:18.950315    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 01:13:19.495967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:19.495356    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:13:19.496195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:19.496108    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:13:19.929475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:19.929383    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused" start-of-body=
Oct 30 01:13:19.929475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:19.929453    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused"
Oct 30 01:13:19.949654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:19.949610    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 01:13:19.950270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:19.950239    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 01:13:20.555274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:20.555227    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:20.929438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:20.929348    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused" start-of-body=
Oct 30 01:13:20.929438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:20.929418    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": dial tcp 10.134.0.188:15021: connect: connection refused"
Oct 30 01:13:20.946352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:20.946323    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused" start-of-body=
Oct 30 01:13:20.946513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:20.946385    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": dial tcp 10.134.0.189:15021: connect: connection refused"
Oct 30 01:13:21.786538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.786457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:21.934651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.934621    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:21.935132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.935108    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:21.935976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.935955    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:13:21.951145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.951122    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:21.951556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.951536    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:21.952165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:21.952147    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:13:22.000800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:22.000747    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podStartSLOduration=9.000734244 podStartE2EDuration="9.000734244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:15.269806957 +0000 UTC m=+1903.299117189" watchObservedRunningTime="2026-10-30 01:13:22.000734244 +0000 UTC m=+1910.030044475"
Oct 30 01:13:22.398267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:22.398218    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:24.213861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:24.213803    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:26.012587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:26.012546    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:27.014775 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:13:27.014737    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:13:27.401518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:27.401430    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"]
Oct 30 01:13:27.401799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:27.401665    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq" podUID="957197fd-0c92-4c61-828a-3eb3e81c6b9d" containerName="maas-api" containerID="cri-o://5d0b7efa84cf3459190ee4c8335b1ce7de5f19240666000462ddebcc0db45893" gracePeriod=30
Oct 30 01:13:27.465903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:27.465862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:28.023871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.013392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:28.132347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.131936    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:28.251860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.251797    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:28.300200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.294437    4577 generic.go:356] "Generic (PLEG): container finished" podID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerID="db3ebf0a5b4239725fbb9f01abf39e472dea9503a717eb68337f49f8066d0573" exitCode=0
Oct 30 01:13:28.300200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.294530    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" event={"ID":"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0","Type":"ContainerDied","Data":"db3ebf0a5b4239725fbb9f01abf39e472dea9503a717eb68337f49f8066d0573"}
Oct 30 01:13:28.300200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.297550    4577 generic.go:356] "Generic (PLEG): container finished" podID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerID="e7e72ac6fad90199fc94b88c9180140f0782ecee80028d00f8c7583f7babe5c8" exitCode=0
Oct 30 01:13:28.300200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.297713    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" event={"ID":"9ccd6567-e409-4ae4-9b4a-be9d8debf002","Type":"ContainerDied","Data":"e7e72ac6fad90199fc94b88c9180140f0782ecee80028d00f8c7583f7babe5c8"}
Oct 30 01:13:28.300200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.299710    4577 generic.go:356] "Generic (PLEG): container finished" podID="957197fd-0c92-4c61-828a-3eb3e81c6b9d" containerID="5d0b7efa84cf3459190ee4c8335b1ce7de5f19240666000462ddebcc0db45893" exitCode=0
Oct 30 01:13:28.300200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.299834    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq" event={"ID":"957197fd-0c92-4c61-828a-3eb3e81c6b9d","Type":"ContainerDied","Data":"5d0b7efa84cf3459190ee4c8335b1ce7de5f19240666000462ddebcc0db45893"}
Oct 30 01:13:28.337015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.332398    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:13:28.427418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.426384    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc\" (UniqueName: \"kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc\") pod \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") "
Oct 30 01:13:28.427418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.426500    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume\") pod \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\" (UID: \"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0\") "
Oct 30 01:13:28.427919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.427864    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume" pod "602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" (UID: "602fe27f-22fd-4b57-ab73-dbbad8e3b6f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:13:28.507974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.507917    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc" pod "602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" (UID: "602fe27f-22fd-4b57-ab73-dbbad8e3b6f0"). InnerVolumeSpecName "kube-api-access-rg5rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:28.553887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.553453    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg5rc\" (UniqueName: \"kubernetes.io/projected/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-kube-api-access-rg5rc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:28.553887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.553509    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:28.591557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.591508    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:13:28.612350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.612305    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"]
Oct 30 01:13:28.613736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.613711    4577 cpu_manager.go:414] "containerMap: removing container" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerName="payload-pre-processing"
Oct 30 01:13:28.613960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.613946    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerName="payload-pre-processing"
Oct 30 01:13:28.614072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.614062    4577 cpu_manager.go:414] "containerMap: removing container" podUID="957197fd-0c92-4c61-828a-3eb3e81c6b9d" containerName="maas-api"
Oct 30 01:13:28.614150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.614141    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="957197fd-0c92-4c61-828a-3eb3e81c6b9d" containerName="maas-api"
Oct 30 01:13:28.614404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.614392    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" containerName="payload-pre-processing"
Oct 30 01:13:28.614507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.614498    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="957197fd-0c92-4c61-828a-3eb3e81c6b9d" containerName="maas-api"
Oct 30 01:13:28.618945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.618924    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.643094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.643068    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:13:28.651067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.651022    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:28.659390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.659060    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"]
Oct 30 01:13:28.783868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.783579    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume\") pod \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") "
Oct 30 01:13:28.783868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.783813    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls\") pod \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") "
Oct 30 01:13:28.784169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.783950    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq\" (UniqueName: \"kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq\") pod \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") "
Oct 30 01:13:28.784169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.784024    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x\" (UniqueName: \"kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x\") pod \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\" (UID: \"9ccd6567-e409-4ae4-9b4a-be9d8debf002\") "
Oct 30 01:13:28.811919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.784320    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs\") pod \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") "
Oct 30 01:13:28.811919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.784387    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca\") pod \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\" (UID: \"957197fd-0c92-4c61-828a-3eb3e81c6b9d\") "
Oct 30 01:13:28.811919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.799079    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume" pod "9ccd6567-e409-4ae4-9b4a-be9d8debf002" (UID: "9ccd6567-e409-4ae4-9b4a-be9d8debf002"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:13:28.811919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.811909    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.812278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.811971    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.812278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.812228    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.812396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.812302    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.812398    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.812636    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.812776    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.812817    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbmk5\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.813168    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.814012    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ccd6567-e409-4ae4-9b4a-be9d8debf002-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:28.814722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.814550    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca" pod "957197fd-0c92-4c61-828a-3eb3e81c6b9d" (UID: "957197fd-0c92-4c61-828a-3eb3e81c6b9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:13:29.008881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.941995    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq" pod "957197fd-0c92-4c61-828a-3eb3e81c6b9d" (UID: "957197fd-0c92-4c61-828a-3eb3e81c6b9d"). InnerVolumeSpecName "kube-api-access-pxmpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:29.008881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.989707    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.008881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.989774    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbmk5\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.008881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.989839    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.008881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.989953    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.008881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:28.989992    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.009522    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.009628    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.009733    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.009812    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.010081    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxmpq\" (UniqueName: \"kubernetes.io/projected/957197fd-0c92-4c61-828a-3eb3e81c6b9d-kube-api-access-pxmpq\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.010108    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957197fd-0c92-4c61-828a-3eb3e81c6b9d-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.010605    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.011646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.011012    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.037870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.012558    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x" pod "9ccd6567-e409-4ae4-9b4a-be9d8debf002" (UID: "9ccd6567-e409-4ae4-9b4a-be9d8debf002"). InnerVolumeSpecName "kube-api-access-w5k6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:29.037870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.012655    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls" pod "957197fd-0c92-4c61-828a-3eb3e81c6b9d" (UID: "957197fd-0c92-4c61-828a-3eb3e81c6b9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:29.037870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.012962    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.037870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.021576    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.037870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.035516    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.039358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.038963    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.040326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.040284    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.040579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.040555    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs" pod "957197fd-0c92-4c61-828a-3eb3e81c6b9d" (UID: "957197fd-0c92-4c61-828a-3eb3e81c6b9d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:29.053760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.052400    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.067977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.067819    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbmk5\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5\") pod \"e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") " pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.111295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.111258    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5k6x\" (UniqueName: \"kubernetes.io/projected/9ccd6567-e409-4ae4-9b4a-be9d8debf002-kube-api-access-w5k6x\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:29.111295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.111301    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:29.111511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.111316    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957197fd-0c92-4c61-828a-3eb3e81c6b9d-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:13:29.273963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.272146    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:29.319430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.319271    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq" event={"ID":"957197fd-0c92-4c61-828a-3eb3e81c6b9d","Type":"ContainerDied","Data":"1527ec87d929f7b8f87dea2d3de253a5f29635287fc9597bbceffa6e72d67c23"}
Oct 30 01:13:29.319767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.319745    4577 scope.go:122] "RemoveContainer" containerID="5d0b7efa84cf3459190ee4c8335b1ce7de5f19240666000462ddebcc0db45893"
Oct 30 01:13:29.320118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.320100    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"
Oct 30 01:13:29.350872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.332415    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8" event={"ID":"602fe27f-22fd-4b57-ab73-dbbad8e3b6f0","Type":"ContainerDied","Data":"c3d3efb241af43614a9a187b805d427b458ea2d33d08f6b6bd118f9eaa3f1608"}
Oct 30 01:13:29.350872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.333509    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"
Oct 30 01:13:29.381880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.378264    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k" event={"ID":"9ccd6567-e409-4ae4-9b4a-be9d8debf002","Type":"ContainerDied","Data":"cc6f6a60ebcd31adc7a4321465ab0500c1f4d0af58a161669983b2033383f877"}
Oct 30 01:13:29.381880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.378391    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"
Oct 30 01:13:29.533659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.533627    4577 scope.go:122] "RemoveContainer" containerID="db3ebf0a5b4239725fbb9f01abf39e472dea9503a717eb68337f49f8066d0573"
Oct 30 01:13:29.557235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.550654    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"]
Oct 30 01:13:29.583820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.575963    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-ce5996-6444d46bbc-znsz8"]
Oct 30 01:13:29.594407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.594331    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"]
Oct 30 01:13:29.601362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.601316    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ce5996-76f5fddc-pqhfq"]
Oct 30 01:13:29.641091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.637055    4577 scope.go:122] "RemoveContainer" containerID="e7e72ac6fad90199fc94b88c9180140f0782ecee80028d00f8c7583f7babe5c8"
Oct 30 01:13:29.657120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.657050    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"]
Oct 30 01:13:29.664128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:29.664094    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-ce5996-6597d55f54-jgb6k"]
Oct 30 01:13:30.266666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.266618    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="602fe27f-22fd-4b57-ab73-dbbad8e3b6f0" path="/var/lib/kubelet/pods/602fe27f-22fd-4b57-ab73-dbbad8e3b6f0/volumes"
Oct 30 01:13:30.268210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.268189    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="957197fd-0c92-4c61-828a-3eb3e81c6b9d" path="/var/lib/kubelet/pods/957197fd-0c92-4c61-828a-3eb3e81c6b9d/volumes"
Oct 30 01:13:30.269093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.269067    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" path="/var/lib/kubelet/pods/9ccd6567-e409-4ae4-9b4a-be9d8debf002/volumes"
Oct 30 01:13:30.520668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.518924    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:30.573872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.565365    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"]
Oct 30 01:13:30.631974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.631923    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:30.632629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.632600    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:30.632813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.632791    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:30.670213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.653152    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:30.700429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:30.700386    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:31.522872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.520948    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 01:13:31.522872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.521579    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:13:31.523478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.523193    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:31.713919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.707658    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 01:13:31.713919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.708405    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:13:31.713919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.711625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:31.758878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.741992    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:31.758878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.742487    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 01:13:31.811924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.810885    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" event={"ID":"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0","Type":"ContainerStarted","Data":"1fb0ed6d9098dcff8ec8a8a24bbdaf7a9d5d50198378581a30b288a00780172a"}
Oct 30 01:13:31.811924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:31.810956    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" event={"ID":"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0","Type":"ContainerStarted","Data":"83ced90f7a19c56a4e7e0f0d9a068b133b7b94e182c1a249097594d12733e54d"}
Oct 30 01:13:32.088427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.088334    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:13:32.095017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.094970    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:32.209894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.201466    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podStartSLOduration=4.20144507 podStartE2EDuration="4.20144507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:32.014336131 +0000 UTC m=+1920.043646386" watchObservedRunningTime="2026-10-30 01:13:32.20144507 +0000 UTC m=+1920.230755302"
Oct 30 01:13:32.345446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.304963    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:32.345446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.306922    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:32.345446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.306999    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:32.655005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.625988    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:32.750914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.628839    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:32.750914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.732893    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:32.751183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.751124    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:32.977923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:32.962238    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:33.355133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:33.355092    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:33.355434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:33.355396    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:33.508939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:33.508909    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:13:33.725472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:33.725386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:33.939324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:33.938864    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 01:13:34.008872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:34.007467    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:13:34.277770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:34.277673    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:34.278093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:34.278061    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:34.417272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:34.417229    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:34.562817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:34.562707    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:35.273869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:35.273230    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:35.273869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:35.273295    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:36.274870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:36.274226    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:36.274870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:36.274287    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:36.442435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:36.442385    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:37.276252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:37.276215    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:37.277914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:37.277882    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:38.273417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:38.273375    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:38.273620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:38.273444    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:38.828731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:38.828701    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:39.273191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:39.273160    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused" start-of-body=
Oct 30 01:13:39.273416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:39.273218    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.190:15021/healthz/ready\": dial tcp 10.134.0.190:15021: connect: connection refused"
Oct 30 01:13:40.307165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.307129    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:40.308053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.308028    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:40.308873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.308836    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:13:40.379270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.379238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:40.493833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.493791    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:40.643914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.643880    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:40.757652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:40.757619    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:42.488553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:42.488519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:44.342058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:44.342020    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:46.230154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:46.230122    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:48.014663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:48.014625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:49.896830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:49.896793    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:51.745275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:51.745234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:53.411029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.410982    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"]
Oct 30 01:13:53.413231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.413153    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerName="payload-processing"
Oct 30 01:13:53.413440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.413423    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerName="payload-processing"
Oct 30 01:13:53.413729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.413715    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ccd6567-e409-4ae4-9b4a-be9d8debf002" containerName="payload-processing"
Oct 30 01:13:53.439994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.439961    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.449496 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.449454    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"]
Oct 30 01:13:53.531264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.531217    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.531564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.531542    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.531905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.531881    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.532093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.532077    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.532263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.532246    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.532420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.532403    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9fff\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.532601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.532582    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.532775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.532762    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.532966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.532950    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634413    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634455    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634491    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634539    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634588    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634748    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634827    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9fff\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.634973 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634955    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.635043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.634975    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.635043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.635026    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.635162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.635075    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.635162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.635147    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.635265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.635228    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.635265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.635229    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.637425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.637398    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.637547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.637531    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.659806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.659737    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.660028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.660006    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9fff\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:53.864876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:53.864806    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:54.235888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:54.235737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:54.309482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:54.297632    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"]
Oct 30 01:13:54.382607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:54.381204    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:54.382607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:54.381301    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:54.382607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:54.381364    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:13:55.384251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:55.384214    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" event={"ID":"ec3a7908-00b6-47ce-9eb9-7d06febf2f49","Type":"ContainerStarted","Data":"e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e"}
Oct 30 01:13:55.384916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:55.384888    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" event={"ID":"ec3a7908-00b6-47ce-9eb9-7d06febf2f49","Type":"ContainerStarted","Data":"e2b5d91b3f576a0f9ced558b04eb1422636e2d569a0f9e77444066c4ebb2d026"}
Oct 30 01:13:55.475455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:55.475399    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podStartSLOduration=2.4753778410000002 podStartE2EDuration="2.475377841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:55.459523692 +0000 UTC m=+1943.488833924" watchObservedRunningTime="2026-10-30 01:13:55.475377841 +0000 UTC m=+1943.504688073"
Oct 30 01:13:55.880872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:55.880489    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:13:55.880872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:55.880620    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:13:55.880872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:55.880674    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:13:56.576365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:56.576327    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:56.739410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:56.739362    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded"
Oct 30 01:13:56.825140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:56.825092    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:13:56.866622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:56.866540    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:13:56.866914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:56.866884    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:13:57.047273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:57.046483    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": context deadline exceeded"
Oct 30 01:13:57.229070 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:13:57.217212    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:13:57.866310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:57.866271    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:13:57.867071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:57.867026    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:13:58.839478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:58.839429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:58.867870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:58.867794    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:13:58.868385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:58.867891    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:13:59.312772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.312742    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:59.726979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.726890    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"]
Oct 30 01:13:59.732157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.732134    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:13:59.756589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.756561    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-0e1a98-openshift-default-dockercfg-tlhv9\""
Oct 30 01:13:59.801484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.801409    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"]
Oct 30 01:13:59.810901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.810801    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:13:59.968987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.968952    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:13:59.981126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.981089    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.989159    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995001    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995059    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995141    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55hrq\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995244    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995331    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995370    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995418    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.007872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:13:59.995456    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.099066    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.099986    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100040    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100099    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100137    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100264    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100307    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100360    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.100444    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55hrq\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.101630    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.102319    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.102571    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.102824    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.115875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.103475    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.130614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.129599    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55hrq\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.130614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.130496    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.134193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.134167    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.143324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.143278    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token\") pod \"e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") " pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.383262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.383166    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:00.409800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.409766    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:00.801336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.801299    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"]
Oct 30 01:14:00.803803 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:00.803765    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9379b6ab_f35d_4d33_bca6_9c9073c290f9.slice/crio-47d92ff8920be8ff639ba2702bd6a172aa44b53289bdad7cd9aa4a609dd85dc5 WatchSource:0}: Error finding container 47d92ff8920be8ff639ba2702bd6a172aa44b53289bdad7cd9aa4a609dd85dc5: Status 404 returned error can't find the container with id 47d92ff8920be8ff639ba2702bd6a172aa44b53289bdad7cd9aa4a609dd85dc5
Oct 30 01:14:00.806261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.806225    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:00.806336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.806295    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:00.806336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.806329    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:00.836410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.836361    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:00.867601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.866969    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:14:00.867601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:00.867030    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:14:01.604734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:01.604692    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" event={"ID":"9379b6ab-f35d-4d33-bca6-9c9073c290f9","Type":"ContainerStarted","Data":"a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"}
Oct 30 01:14:01.605198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:01.604783    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" event={"ID":"9379b6ab-f35d-4d33-bca6-9c9073c290f9","Type":"ContainerStarted","Data":"47d92ff8920be8ff639ba2702bd6a172aa44b53289bdad7cd9aa4a609dd85dc5"}
Oct 30 01:14:01.678504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:01.678446    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podStartSLOduration=2.678431018 podStartE2EDuration="2.678431018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:01.669266665 +0000 UTC m=+1949.698576898" watchObservedRunningTime="2026-10-30 01:14:01.678431018 +0000 UTC m=+1949.707741249"
Oct 30 01:14:01.878917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:01.868290    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused" start-of-body=
Oct 30 01:14:01.878917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:01.868352    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.191:15021/healthz/ready\": dial tcp 10.134.0.191:15021: connect: connection refused"
Oct 30 01:14:02.410922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:02.410884    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:02.416969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:02.416943    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:02.613807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:02.613777    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:02.614916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:02.614892    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:14:02.703349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:02.703252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:03.334396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:03.334368    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:14:03.335304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:03.335282    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:14:03.335629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:03.335614    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:14:04.048132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:04.048101    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:04.183865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:04.183825    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:04.715259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:04.715219    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:05.748443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:05.748406    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:05.865282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:05.865253    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:06.612078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:06.612046    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.033383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.033346    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"]
Oct 30 01:14:08.034241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.034208    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n" podUID="44183be9-c26d-4f52-bc5f-f7a65dfadf14" containerName="maas-api" containerID="cri-o://797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a" gracePeriod=30
Oct 30 01:14:08.112519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.112444    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.200546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.200507    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"]
Oct 30 01:14:08.200785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.200758    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm" podUID="25441484-890b-4e6c-94e0-b4dd99376b8c" containerName="payload-processing" containerID="cri-o://ea5d7ce7fa74a8ee53b6777374250ce76e132009022b89f28d9dcbc9ea5c99b4" gracePeriod=30
Oct 30 01:14:08.226639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.226591    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"]
Oct 30 01:14:08.226889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.226862    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654" podUID="d06b60da-d546-4ce9-ac26-7b38882a18d5" containerName="payload-pre-processing" containerID="cri-o://e941a2c409b5927c58789cd26180991ec1a36f5257badb70819a2be1f3b71c6e" gracePeriod=30
Oct 30 01:14:08.281003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.280967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.401135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.401094    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"]
Oct 30 01:14:08.402788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.402761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.406510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.406494    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.430365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.429955    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:14:08.434991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.434837    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"]
Oct 30 01:14:08.711922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.707245    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs\") pod \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") "
Oct 30 01:14:08.747801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.710165    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.748163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.748140    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl\" (UniqueName: \"kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl\") pod \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") "
Oct 30 01:14:08.748392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.748371    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls\") pod \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") "
Oct 30 01:14:08.748504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.748432    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca\") pod \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\" (UID: \"44183be9-c26d-4f52-bc5f-f7a65dfadf14\") "
Oct 30 01:14:08.748753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.748729    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749051    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749190    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749311    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749567    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749678    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749709    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749836    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.749954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.749942    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pl4d\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.752040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.751985    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca" pod "44183be9-c26d-4f52-bc5f-f7a65dfadf14" (UID: "44183be9-c26d-4f52-bc5f-f7a65dfadf14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:08.756034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.755996    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl" pod "44183be9-c26d-4f52-bc5f-f7a65dfadf14" (UID: "44183be9-c26d-4f52-bc5f-f7a65dfadf14"). InnerVolumeSpecName "kube-api-access-c28jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:08.758890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.758838    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs" pod "44183be9-c26d-4f52-bc5f-f7a65dfadf14" (UID: "44183be9-c26d-4f52-bc5f-f7a65dfadf14"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:14:08.759224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.759192    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls" pod "44183be9-c26d-4f52-bc5f-f7a65dfadf14" (UID: "44183be9-c26d-4f52-bc5f-f7a65dfadf14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:14:08.793834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.793799    4577 generic.go:356] "Generic (PLEG): container finished" podID="25441484-890b-4e6c-94e0-b4dd99376b8c" containerID="ea5d7ce7fa74a8ee53b6777374250ce76e132009022b89f28d9dcbc9ea5c99b4" exitCode=0
Oct 30 01:14:08.794035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.794002    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm" event={"ID":"25441484-890b-4e6c-94e0-b4dd99376b8c","Type":"ContainerDied","Data":"ea5d7ce7fa74a8ee53b6777374250ce76e132009022b89f28d9dcbc9ea5c99b4"}
Oct 30 01:14:08.796426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.796315    4577 generic.go:356] "Generic (PLEG): container finished" podID="d06b60da-d546-4ce9-ac26-7b38882a18d5" containerID="e941a2c409b5927c58789cd26180991ec1a36f5257badb70819a2be1f3b71c6e" exitCode=0
Oct 30 01:14:08.796426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.796384    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654" event={"ID":"d06b60da-d546-4ce9-ac26-7b38882a18d5","Type":"ContainerDied","Data":"e941a2c409b5927c58789cd26180991ec1a36f5257badb70819a2be1f3b71c6e"}
Oct 30 01:14:08.802741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.797988    4577 generic.go:356] "Generic (PLEG): container finished" podID="44183be9-c26d-4f52-bc5f-f7a65dfadf14" containerID="797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a" exitCode=0
Oct 30 01:14:08.802741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.798026    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n" event={"ID":"44183be9-c26d-4f52-bc5f-f7a65dfadf14","Type":"ContainerDied","Data":"797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a"}
Oct 30 01:14:08.802741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.798054    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n" event={"ID":"44183be9-c26d-4f52-bc5f-f7a65dfadf14","Type":"ContainerDied","Data":"d8b06db6281f0fc85ac08ddf94b0738053c446ccab86717a3f736388228df5a9"}
Oct 30 01:14:08.802741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.798075    4577 scope.go:122] "RemoveContainer" containerID="797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a"
Oct 30 01:14:08.802741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.798193    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"
Oct 30 01:14:08.808250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.808112    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:14:08.813759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.813740    4577 scope.go:122] "RemoveContainer" containerID="797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a"
Oct 30 01:14:08.814186 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:14:08.814157    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a\": container with ID starting with 797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a not found: ID does not exist" containerID="797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a"
Oct 30 01:14:08.814295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.814194    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a"} err="failed to get container status \"797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a\": rpc error: code = NotFound desc = could not find container \"797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a\": container with ID starting with 797e1ce87ff764d4c2f5c137a5fe7336765d871ae50b94535afaa206669ef81a not found: ID does not exist"
Oct 30 01:14:08.824078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.824057    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:14:08.851303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851276    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851312    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851328    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851418    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851463    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pl4d\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851516    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851652    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851678    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851709    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.851936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851891    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c28jl\" (UniqueName: \"kubernetes.io/projected/44183be9-c26d-4f52-bc5f-f7a65dfadf14-kube-api-access-c28jl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:08.851936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851907    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:08.851936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851922    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44183be9-c26d-4f52-bc5f-f7a65dfadf14-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:08.852090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.851936    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44183be9-c26d-4f52-bc5f-f7a65dfadf14-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:08.852140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.852097    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.852218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.852199    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.852444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.852427    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.852709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.852688    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.853113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.853079    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.855138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.855116    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.855244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.855115    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.870433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.870399    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.877824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.877797    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.878024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.877946    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pl4d\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d\") pod \"e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") " pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:08.904718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.904665    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"]
Oct 30 01:14:08.908438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.908414    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d3c1fa-8689998f9f-mbj7n"]
Oct 30 01:14:08.976969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.955537    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq\" (UniqueName: \"kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq\") pod \"d06b60da-d546-4ce9-ac26-7b38882a18d5\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") "
Oct 30 01:14:08.976969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.955767    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume\") pod \"d06b60da-d546-4ce9-ac26-7b38882a18d5\" (UID: \"d06b60da-d546-4ce9-ac26-7b38882a18d5\") "
Oct 30 01:14:08.976969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.955827    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf\" (UniqueName: \"kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf\") pod \"25441484-890b-4e6c-94e0-b4dd99376b8c\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") "
Oct 30 01:14:08.976969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.956052    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume\") pod \"25441484-890b-4e6c-94e0-b4dd99376b8c\" (UID: \"25441484-890b-4e6c-94e0-b4dd99376b8c\") "
Oct 30 01:14:08.991893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.977773    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume" pod "25441484-890b-4e6c-94e0-b4dd99376b8c" (UID: "25441484-890b-4e6c-94e0-b4dd99376b8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:08.991893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.987862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:08.992493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:08.992430    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume" pod "d06b60da-d546-4ce9-ac26-7b38882a18d5" (UID: "d06b60da-d546-4ce9-ac26-7b38882a18d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:09.004123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.003923    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq" pod "d06b60da-d546-4ce9-ac26-7b38882a18d5" (UID: "d06b60da-d546-4ce9-ac26-7b38882a18d5"). InnerVolumeSpecName "kube-api-access-zhqvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:09.004733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.004679    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf" pod "25441484-890b-4e6c-94e0-b4dd99376b8c" (UID: "25441484-890b-4e6c-94e0-b4dd99376b8c"). InnerVolumeSpecName "kube-api-access-vghrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:09.055822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.055794    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:09.058594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.058569    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d06b60da-d546-4ce9-ac26-7b38882a18d5-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:09.058594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.058595    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vghrf\" (UniqueName: \"kubernetes.io/projected/25441484-890b-4e6c-94e0-b4dd99376b8c-kube-api-access-vghrf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:09.058754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.058605    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25441484-890b-4e6c-94e0-b4dd99376b8c-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:09.058754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.058615    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhqvq\" (UniqueName: \"kubernetes.io/projected/d06b60da-d546-4ce9-ac26-7b38882a18d5-kube-api-access-zhqvq\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:09.347660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.347619    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:09.381520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.381488    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"]
Oct 30 01:14:09.382205 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:09.382175    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6c8b03_8c63_4741_93f0_faa89aee5aac.slice/crio-80e0455817d3ab9ed0211f749eece42b1172e6e445a7052d282de12fa3ef70c1 WatchSource:0}: Error finding container 80e0455817d3ab9ed0211f749eece42b1172e6e445a7052d282de12fa3ef70c1: Status 404 returned error can't find the container with id 80e0455817d3ab9ed0211f749eece42b1172e6e445a7052d282de12fa3ef70c1
Oct 30 01:14:09.384820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.384781    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:09.384978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.384879    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:09.384978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.384945    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:09.498756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.498725    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:09.808757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.808712    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" event={"ID":"ef6c8b03-8c63-4741-93f0-faa89aee5aac","Type":"ContainerStarted","Data":"f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566"}
Oct 30 01:14:09.808992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.808784    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" event={"ID":"ef6c8b03-8c63-4741-93f0-faa89aee5aac","Type":"ContainerStarted","Data":"80e0455817d3ab9ed0211f749eece42b1172e6e445a7052d282de12fa3ef70c1"}
Oct 30 01:14:09.810467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.810442    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"
Oct 30 01:14:09.810604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.810475    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm" event={"ID":"25441484-890b-4e6c-94e0-b4dd99376b8c","Type":"ContainerDied","Data":"d5584962f1132b1a4ef8ee16334b380889ef8fce09da02470d3d9fffd2c142cb"}
Oct 30 01:14:09.810604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.810534    4577 scope.go:122] "RemoveContainer" containerID="ea5d7ce7fa74a8ee53b6777374250ce76e132009022b89f28d9dcbc9ea5c99b4"
Oct 30 01:14:09.815993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.815958    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654" event={"ID":"d06b60da-d546-4ce9-ac26-7b38882a18d5","Type":"ContainerDied","Data":"90e898f4e88b3d4fb5fe4b4c5374546e48d17197a2f4da1629f36a253a1e831f"}
Oct 30 01:14:09.816099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.815966    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"
Oct 30 01:14:09.832489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.832453    4577 scope.go:122] "RemoveContainer" containerID="e941a2c409b5927c58789cd26180991ec1a36f5257badb70819a2be1f3b71c6e"
Oct 30 01:14:09.862294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.862241    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podStartSLOduration=1.8622232250000001 podStartE2EDuration="1.862223225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:09.857591393 +0000 UTC m=+1957.886901626" watchObservedRunningTime="2026-10-30 01:14:09.862223225 +0000 UTC m=+1957.891533455"
Oct 30 01:14:09.899866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.899338    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"]
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900298    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d06b60da-d546-4ce9-ac26-7b38882a18d5" containerName="payload-pre-processing"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900320    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d06b60da-d546-4ce9-ac26-7b38882a18d5" containerName="payload-pre-processing"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900387    4577 cpu_manager.go:414] "containerMap: removing container" podUID="44183be9-c26d-4f52-bc5f-f7a65dfadf14" containerName="maas-api"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900395    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44183be9-c26d-4f52-bc5f-f7a65dfadf14" containerName="maas-api"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900422    4577 cpu_manager.go:414] "containerMap: removing container" podUID="25441484-890b-4e6c-94e0-b4dd99376b8c" containerName="payload-processing"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900429    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25441484-890b-4e6c-94e0-b4dd99376b8c" containerName="payload-processing"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900542    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="25441484-890b-4e6c-94e0-b4dd99376b8c" containerName="payload-processing"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900558    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d06b60da-d546-4ce9-ac26-7b38882a18d5" containerName="payload-pre-processing"
Oct 30 01:14:09.903869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.900578    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="44183be9-c26d-4f52-bc5f-f7a65dfadf14" containerName="maas-api"
Oct 30 01:14:09.931124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.931081    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"]
Oct 30 01:14:09.931124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.931125    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d3c1fa-7b4cf776d-ckpbm"]
Oct 30 01:14:09.931385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.931220    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:09.931451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.931379    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"]
Oct 30 01:14:09.990106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.988932    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-8ae41e-openshift-default-dockercfg-22s5p\""
Oct 30 01:14:09.990106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.988931    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:09.996862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:09.996810    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"]
Oct 30 01:14:10.087404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.087313    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:10.102871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.090888    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:10.102871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.090955    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:10.102871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.101900    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-d3c1fa-5dd9c4bf4c-vm654"]
Oct 30 01:14:10.103190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103024    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103110    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103153    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103191    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103322    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103417    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103456    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103496    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.103821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.103673    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.213818    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.213908    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.213950    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.214007    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.214078    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.214139    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.214175    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.214214    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.214870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.214342    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.215530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.215237    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.215591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.215524    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.229607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.218340    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.233700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.230863    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.233931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.231112    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.263887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.257644    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.267444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.267411    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25441484-890b-4e6c-94e0-b4dd99376b8c" path="/var/lib/kubelet/pods/25441484-890b-4e6c-94e0-b4dd99376b8c/volumes"
Oct 30 01:14:10.273501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.273461    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44183be9-c26d-4f52-bc5f-f7a65dfadf14" path="/var/lib/kubelet/pods/44183be9-c26d-4f52-bc5f-f7a65dfadf14/volumes"
Oct 30 01:14:10.287476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.287446    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d06b60da-d546-4ce9-ac26-7b38882a18d5" path="/var/lib/kubelet/pods/d06b60da-d546-4ce9-ac26-7b38882a18d5/volumes"
Oct 30 01:14:10.287947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.285564    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.300403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.300371    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.316876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.301934    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo\") pod \"e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") " pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.492200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.492161    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:10.604165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.604109    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:10.953010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:10.951461    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:11.023508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.022697    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:11.023508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.022797    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:11.023508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.022860    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:11.035126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.034344    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"]
Oct 30 01:14:11.084290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.084254    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:11.084550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.084521    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:11.085645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.085619    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:11.204310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.204194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:11.603003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.602968    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"]
Oct 30 01:14:11.608284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.608258    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.632151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.630813    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-df4db1-openshift-default-dockercfg-49p7w\""
Oct 30 01:14:11.679057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.678982    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:11.679835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.679427    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"]
Oct 30 01:14:11.732055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732017    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732092    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732151    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732336    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732427    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732492    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtbkj\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732552    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732606    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.732664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.732648    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.793589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.793553    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:11.844793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.844745    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.850132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.850100    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.850403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.850383    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.851927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.851900    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtbkj\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.852208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.852130    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.852352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.852334    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.852491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.852475    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.858870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.858794    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.859079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.859059    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.859288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.853681    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.859383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.858373    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.859471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.853258    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.863922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.860366    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.864076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.853484    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.864158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.860617    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.903904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.890665    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.926871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.921478    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:11.967109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:11.929713    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtbkj\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj\") pod \"e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") " pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:12.128318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.128214    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:12.133787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.133749    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:12.134333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.134302    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:12.165227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.165161    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" event={"ID":"38c3a078-0da3-44e6-91b9-32604ce48f3d","Type":"ContainerStarted","Data":"ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"}
Oct 30 01:14:12.167506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.167477    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" event={"ID":"38c3a078-0da3-44e6-91b9-32604ce48f3d","Type":"ContainerStarted","Data":"7050ae6b7b01682e4243f4782aa6d313890cacae48bfe02da09c6fc808afb99a"}
Oct 30 01:14:12.229946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.229909    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:12.235194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.235153    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podStartSLOduration=3.2351403579999998 podStartE2EDuration="3.235140358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:12.233774301 +0000 UTC m=+1960.263084529" watchObservedRunningTime="2026-10-30 01:14:12.235140358 +0000 UTC m=+1960.264450588"
Oct 30 01:14:12.585186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.585153    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"]
Oct 30 01:14:12.585328 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:12.585220    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b1183c_5416_461c_9375_7e377bb762a1.slice/crio-e4380bcdf261505a97ee1a206a60aea81c960248c4cd36e3942e7086a5328539 WatchSource:0}: Error finding container e4380bcdf261505a97ee1a206a60aea81c960248c4cd36e3942e7086a5328539: Status 404 returned error can't find the container with id e4380bcdf261505a97ee1a206a60aea81c960248c4cd36e3942e7086a5328539
Oct 30 01:14:12.588047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.588017    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:12.588148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.588081    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:12.588148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.588109    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:12.604539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.604511    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:12.611975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.611682    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:12.819726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:12.819683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:13.066873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.061338    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:13.066873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.061412    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:13.194297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.194259    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" event={"ID":"c4b1183c-5416-461c-9375-7e377bb762a1","Type":"ContainerStarted","Data":"5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"}
Oct 30 01:14:13.196592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.196562    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" event={"ID":"c4b1183c-5416-461c-9375-7e377bb762a1","Type":"ContainerStarted","Data":"e4380bcdf261505a97ee1a206a60aea81c960248c4cd36e3942e7086a5328539"}
Oct 30 01:14:13.196797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.196782    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:13.196987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.196973    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:14:13.230643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.230614    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:13.235884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.235828    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:13.264163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:13.264100    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podStartSLOduration=2.264078403 podStartE2EDuration="2.264078403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:13.255355807 +0000 UTC m=+1961.284666044" watchObservedRunningTime="2026-10-30 01:14:13.264078403 +0000 UTC m=+1961.293388646"
Oct 30 01:14:14.063873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:14.059200    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:14.063873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:14.059258    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:14.199485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:14.199447    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:14.200422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:14.200399    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:14:14.721792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:14.721752    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:14.839474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:14.839440    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:15.067870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:15.061779    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:15.067870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:15.061899    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:15.668868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:15.662144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:15.790820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:15.790777    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:16.056570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.056530    4577 patch_prober.go:29] interesting pod/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused" start-of-body=
Oct 30 01:14:16.056779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.056605    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.193:15021/healthz/ready\": dial tcp 10.134.0.193:15021: connect: connection refused"
Oct 30 01:14:16.486926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.486892    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:16.602960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.602930    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:16.874927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.863980    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"]
Oct 30 01:14:16.997871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.987464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:16.997871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:16.988200    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" containerID="cri-o://19aefa00ad9669bde87b9ffc79f7b7641e0bca2f0bfd96e9e625ea589b97d72c" gracePeriod=30
Oct 30 01:14:17.358910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:17.338370    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:17.727637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:17.727555    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:17.727996 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:17.727964    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:17.728822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:17.728797    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:14:17.994698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:17.994665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:18.205402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.205370    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:18.329502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.329413    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:18.334825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.334789    4577 patch_prober.go:29] interesting pod/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:14:18.334994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.334866    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:14:18.404419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.403935    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"]
Oct 30 01:14:18.410186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.410161    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.426209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.426179    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"]
Oct 30 01:14:18.456113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.456077    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4nb9\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.456487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.456466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.456713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.456696    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.456887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.456873    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.457048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.457030    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.457271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.457252    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.457534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.457506    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.457747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.457729    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.457995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.457978    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.490715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.490682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:18.491345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.491325    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-35042a-openshift-default-dockercfg-m2vz5\""
Oct 30 01:14:18.588479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.588073    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.589423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.588775    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.594887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.589265    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.595498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.595476    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.596003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.595984    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.596494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.596477    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.603775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.596138    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.604928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.603997    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.604928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.604065    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4nb9\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.604928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.604115    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.604928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.604248    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.606072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.606054    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.607103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.607083    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.607662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.607388    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.640871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.635492    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.640871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.638471    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.644873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.644626    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4nb9\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.645874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.645066    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token\") pod \"e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") " pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.742881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.742208    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:18.900519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:18.900432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:19.097006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.096695    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:19.194156 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:19.194128    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1e68a2_1048_47ae_a00e_3fa2f63fa45d.slice/crio-e734ac2ae389a567cbe7bc5e1b703871fa5de5287774f65e4fba55d566b2e54b WatchSource:0}: Error finding container e734ac2ae389a567cbe7bc5e1b703871fa5de5287774f65e4fba55d566b2e54b: Status 404 returned error can't find the container with id e734ac2ae389a567cbe7bc5e1b703871fa5de5287774f65e4fba55d566b2e54b
Oct 30 01:14:19.194156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.194141    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"]
Oct 30 01:14:19.196726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.196697    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:19.196809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.196756    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:19.196809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.196786    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:19.257605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.257567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:19.457151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.457059    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" event={"ID":"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d","Type":"ContainerStarted","Data":"2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"}
Oct 30 01:14:19.457151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.457131    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" event={"ID":"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d","Type":"ContainerStarted","Data":"e734ac2ae389a567cbe7bc5e1b703871fa5de5287774f65e4fba55d566b2e54b"}
Oct 30 01:14:19.508642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.508589    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podStartSLOduration=1.508574772 podStartE2EDuration="1.508574772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:19.50517524 +0000 UTC m=+1967.534485471" watchObservedRunningTime="2026-10-30 01:14:19.508574772 +0000 UTC m=+1967.537885003"
Oct 30 01:14:19.743095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.743065    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:19.911371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:19.911336    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:20.050925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:20.045207    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:20.259277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:20.259232    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"]
Oct 30 01:14:20.273299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:20.273262    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy" containerID="cri-o://5f4c28e3754229d9a3646e0363609688b96beda080c069b38dce295be6920c9b" gracePeriod=30
Oct 30 01:14:20.278128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:20.277398    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:20.787872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:20.777289    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:21.193062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:21.192945    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:21.733871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:21.733583    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:21.745604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:21.745276    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:14:21.991158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:21.991035    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:22.351944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:22.335428    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:22.536872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:22.523407    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:22.793872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:22.780979    4577 generic.go:356] "Generic (PLEG): container finished" podID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerID="19aefa00ad9669bde87b9ffc79f7b7641e0bca2f0bfd96e9e625ea589b97d72c" exitCode=0
Oct 30 01:14:22.793872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:22.782379    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" event={"ID":"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6","Type":"ContainerDied","Data":"19aefa00ad9669bde87b9ffc79f7b7641e0bca2f0bfd96e9e625ea589b97d72c"}
Oct 30 01:14:22.793872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:22.782459    4577 scope.go:122] "RemoveContainer" containerID="89e78379fa5969b5f067c26c1b09f7cb06513e5b7cc1b43ac9faec57aeb93acd"
Oct 30 01:14:23.153489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.151954    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.188:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:14:23.153489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.152028    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.188:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:14:23.208793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.208758    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.189:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:14:23.209083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.209054    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.189:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:14:23.670160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.670070    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:23.699902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.693349    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.810723    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.810802    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.810869    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.810982    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.811080    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.811154    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.811242    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.811338    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.811407    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket\") pod \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\" (UID: \"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6\") "
Oct 30 01:14:23.826272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.824740    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:23.826272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.825432    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:23.826272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.825645    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:23.837370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.836199    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"
Oct 30 01:14:23.837370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.836556    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp" event={"ID":"df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6","Type":"ContainerDied","Data":"9720eb454b6d894dd30ba57eb78bf0cd416c66fa5796dfa5d8e0499785c8efdf"}
Oct 30 01:14:23.837370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.836603    4577 scope.go:122] "RemoveContainer" containerID="19aefa00ad9669bde87b9ffc79f7b7641e0bca2f0bfd96e9e625ea589b97d72c"
Oct 30 01:14:23.845491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.838604    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:23.845491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.841411    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:23.856763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.849951    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:23.856763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.855052    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:23.859252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.858074    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:14:23.892251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.883937    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8" pod "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" (UID: "df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6"). InnerVolumeSpecName "kube-api-access-n8ql8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:23.915558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915508    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.915558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915547    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.915558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915561    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.916011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915574    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8ql8\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-kube-api-access-n8ql8\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.916011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915589    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.916011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915603    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.916011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915615    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.916011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915629    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.916011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.915645    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:23.943794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:23.943688    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:24.055737 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.055702    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:24.323971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.323934    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"]
Oct 30 01:14:24.456137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.456106    4577 cpu_manager.go:414] "containerMap: removing container" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy"
Oct 30 01:14:24.456366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.456351    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy"
Oct 30 01:14:24.457000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.456976    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy"
Oct 30 01:14:24.457175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.457162    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy"
Oct 30 01:14:24.476225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.476199    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"]
Oct 30 01:14:24.476412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.476400    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d3c1fa-openshift-default-5bf5b6f4df-xhdzp"]
Oct 30 01:14:24.476513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.476505    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"]
Oct 30 01:14:24.476588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.476426    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.493361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.493331    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-35042a-dockercfg-k57gv\""
Oct 30 01:14:24.493361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.493332    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-35042a\""
Oct 30 01:14:24.612994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.612903    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qw7q\" (UniqueName: \"kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q\") pod \"payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.613596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.613130    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.714067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.714032    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.714237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.714148    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qw7q\" (UniqueName: \"kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q\") pod \"payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.714706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.714686    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.739573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.739548    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qw7q\" (UniqueName: \"kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q\") pod \"payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:24.792402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:24.792371    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:25.100901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.100871    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"]
Oct 30 01:14:25.101514 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:25.101481    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421abe2b_d2f3_4455_81c9_1ac65acedd8f.slice/crio-4ea81f13cfa3a5b014f874bc7a784c3dcd17c16f9ab560da324dcb1b14863661 WatchSource:0}: Error finding container 4ea81f13cfa3a5b014f874bc7a784c3dcd17c16f9ab560da324dcb1b14863661: Status 404 returned error can't find the container with id 4ea81f13cfa3a5b014f874bc7a784c3dcd17c16f9ab560da324dcb1b14863661
Oct 30 01:14:25.601804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.601769    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:25.861035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.860959    4577 generic.go:356] "Generic (PLEG): container finished" podID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerID="5f4c28e3754229d9a3646e0363609688b96beda080c069b38dce295be6920c9b" exitCode=0
Oct 30 01:14:25.861208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.861033    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" event={"ID":"c1a46170-e759-428e-aa74-dbf88c1fdd26","Type":"ContainerDied","Data":"5f4c28e3754229d9a3646e0363609688b96beda080c069b38dce295be6920c9b"}
Oct 30 01:14:25.861208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.861111    4577 scope.go:122] "RemoveContainer" containerID="13321903ceee490d5a1bc4039a0bd595312560850b81d3c17c162d0f582b2052"
Oct 30 01:14:25.863108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.863082    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" event={"ID":"421abe2b-d2f3-4455-81c9-1ac65acedd8f","Type":"ContainerStarted","Data":"c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28"}
Oct 30 01:14:25.863230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.863136    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" event={"ID":"421abe2b-d2f3-4455-81c9-1ac65acedd8f","Type":"ContainerStarted","Data":"4ea81f13cfa3a5b014f874bc7a784c3dcd17c16f9ab560da324dcb1b14863661"}
Oct 30 01:14:25.863440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.863422    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:25.909540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.909483    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" podStartSLOduration=1.909470673 podStartE2EDuration="1.909470673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:25.90710902 +0000 UTC m=+1973.936419264" watchObservedRunningTime="2026-10-30 01:14:25.909470673 +0000 UTC m=+1973.938780929"
Oct 30 01:14:25.915004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:25.914973    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035392    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035495    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035542    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035640    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035708    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035782    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035913    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035953    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.040404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.035981    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert\") pod \"c1a46170-e759-428e-aa74-dbf88c1fdd26\" (UID: \"c1a46170-e759-428e-aa74-dbf88c1fdd26\") "
Oct 30 01:14:26.042702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.042644    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:26.044258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.043637    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:26.044405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.044367    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:26.061467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.048949    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:26.082393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.075226    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:14:26.082393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.081069    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:26.098008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.097970    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:14:26.098456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.098416    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "kube-api-access-jbbj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:26.102907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.099301    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token" pod "c1a46170-e759-428e-aa74-dbf88c1fdd26" (UID: "c1a46170-e759-428e-aa74-dbf88c1fdd26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:26.119296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.119233    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" path="/var/lib/kubelet/pods/df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6/volumes"
Oct 30 01:14:26.138135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138110    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138136    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138146    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138156    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbbj2\" (UniqueName: \"kubernetes.io/projected/c1a46170-e759-428e-aa74-dbf88c1fdd26-kube-api-access-jbbj2\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138164    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138172    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1a46170-e759-428e-aa74-dbf88c1fdd26-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138180    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138187    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.138282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.138195    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1a46170-e759-428e-aa74-dbf88c1fdd26-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:26.871759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.871734    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"
Oct 30 01:14:26.872001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.871774    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh" event={"ID":"c1a46170-e759-428e-aa74-dbf88c1fdd26","Type":"ContainerDied","Data":"f0e1e74fa7f0b09d7c613a198985705b2ac0bac20c7bfa4480aa0b39dabf31bd"}
Oct 30 01:14:26.872001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.871858    4577 scope.go:122] "RemoveContainer" containerID="5f4c28e3754229d9a3646e0363609688b96beda080c069b38dce295be6920c9b"
Oct 30 01:14:26.946025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.945996    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"]
Oct 30 01:14:26.949376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:26.949353    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-ce5996-openshift-default-5969c7484f-s8tfh"]
Oct 30 01:14:27.015115 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:14:27.015077    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:14:27.408986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:27.408952    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:28.111918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:28.111888    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" path="/var/lib/kubelet/pods/c1a46170-e759-428e-aa74-dbf88c1fdd26/volumes"
Oct 30 01:14:29.209800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:29.209761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:30.881180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.881149    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"]
Oct 30 01:14:30.882087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.882066    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy"
Oct 30 01:14:30.882173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.882090    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy"
Oct 30 01:14:30.882173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.882113    4577 cpu_manager.go:414] "containerMap: removing container" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy"
Oct 30 01:14:30.882173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.882121    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df49c3e7-04b3-4ca8-8e90-4dcd078bfbf6" containerName="istio-proxy"
Oct 30 01:14:30.882291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.882280    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy"
Oct 30 01:14:30.882327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.882300    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy"
Oct 30 01:14:30.888351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.888329    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:30.907053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.907021    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"]
Oct 30 01:14:30.912582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.912547    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-8ae41e-serving-cert\""
Oct 30 01:14:30.941624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.941593    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:30.941802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.941659    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:30.941901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.941807    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s7lm\" (UniqueName: \"kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:30.941964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.941898    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:30.971277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:30.971242    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:31.044910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.044873    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.045089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.044954    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.045089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.045043    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s7lm\" (UniqueName: \"kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.045175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.045104    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.045791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.045703    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.047648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.047626    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.047648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.047632    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.071386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.071361    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s7lm\" (UniqueName: \"kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm\") pod \"maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.082352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.082290    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:31.196916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.193337    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:31.216876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.215005    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:31.528321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.528255    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:31.550341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.550313    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"]
Oct 30 01:14:31.550466 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:31.550331    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142ed92b_5389_4bff_96ba_683474be650f.slice/crio-d16c910b580fb5c9f63aa9bfa475ad6ef1e399840f3145df86f4f58252461513 WatchSource:0}: Error finding container d16c910b580fb5c9f63aa9bfa475ad6ef1e399840f3145df86f4f58252461513: Status 404 returned error can't find the container with id d16c910b580fb5c9f63aa9bfa475ad6ef1e399840f3145df86f4f58252461513
Oct 30 01:14:31.637824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.637732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:31.965994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:31.958218    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj" event={"ID":"142ed92b-5389-4bff-96ba-683474be650f","Type":"ContainerStarted","Data":"d16c910b580fb5c9f63aa9bfa475ad6ef1e399840f3145df86f4f58252461513"}
Oct 30 01:14:32.765662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.765628    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"]
Oct 30 01:14:32.766342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.766326    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy"
Oct 30 01:14:32.766412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.766345    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1a46170-e759-428e-aa74-dbf88c1fdd26" containerName="istio-proxy"
Oct 30 01:14:32.770105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.770088    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.791295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.791265    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"]
Oct 30 01:14:32.804934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.800365    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-df4db1-serving-cert\""
Oct 30 01:14:32.859253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.859220    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.859938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.859715    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.859938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.859900    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhpkx\" (UniqueName: \"kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.860153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.859947    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.866459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.866429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:32.970905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.964803    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.970905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.964974    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhpkx\" (UniqueName: \"kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.970905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.965037    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.970905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.965100    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.970905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.969108    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.970905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.969939    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.997875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.988177    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:32.997875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:32.993988    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:33.019970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:33.014696    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhpkx\" (UniqueName: \"kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx\") pod \"maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:33.071349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:33.070436    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj" event={"ID":"142ed92b-5389-4bff-96ba-683474be650f","Type":"ContainerStarted","Data":"a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a"}
Oct 30 01:14:33.071704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:33.071562    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:33.085959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:33.085928    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:33.116604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:33.116554    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj" podStartSLOduration=2.787891175 podStartE2EDuration="3.116539605s" totalImagesPullingTime="328.64843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:33.116361633 +0000 UTC m=+1981.145671958" watchObservedRunningTime="2026-10-30 01:14:33.116539605 +0000 UTC m=+1981.145849883"
Oct 30 01:14:33.393487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:33.393395    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"]
Oct 30 01:14:33.394950 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:33.394918    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38283fe4_8bfc_442b_b730_87b614feed0c.slice/crio-5d5e311db776844daea82ef523b0a2fb99d0b154f7db0cd82f7719aebde31149 WatchSource:0}: Error finding container 5d5e311db776844daea82ef523b0a2fb99d0b154f7db0cd82f7719aebde31149: Status 404 returned error can't find the container with id 5d5e311db776844daea82ef523b0a2fb99d0b154f7db0cd82f7719aebde31149
Oct 30 01:14:34.079997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:34.079896    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft" event={"ID":"38283fe4-8bfc-442b-b730-87b614feed0c","Type":"ContainerStarted","Data":"9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97"}
Oct 30 01:14:34.079997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:34.079947    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft" event={"ID":"38283fe4-8bfc-442b-b730-87b614feed0c","Type":"ContainerStarted","Data":"5d5e311db776844daea82ef523b0a2fb99d0b154f7db0cd82f7719aebde31149"}
Oct 30 01:14:34.080518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:34.080247    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:34.133357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:34.133295    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft" podStartSLOduration=1.785988239 podStartE2EDuration="2.133282355s" totalImagesPullingTime="347.294116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:34.130916676 +0000 UTC m=+1982.160226907" watchObservedRunningTime="2026-10-30 01:14:34.133282355 +0000 UTC m=+1982.162592626"
Oct 30 01:14:34.626591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:34.626532    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:36.538061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:36.538028    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:36.875781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:36.875702    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:14:38.316804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:38.316765    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:40.059147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.059108    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"]
Oct 30 01:14:40.066609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.066578    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.084401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.084370    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"]
Oct 30 01:14:40.093670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.093642    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-35042a-serving-cert\""
Oct 30 01:14:40.096528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.096508    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:14:40.097801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.097783    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:14:40.161324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.161287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:40.214425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.214387    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.214587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.214561    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f7zl\" (UniqueName: \"kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.214663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.214646    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.214952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.214921    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.289233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.289191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:40.316304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.316229    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f7zl\" (UniqueName: \"kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.316304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.316290    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.316525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.316347    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.316525 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:14:40.316459    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-serving-cert: secret "maas-api-e2e-shared-a-w2-35042a-serving-cert" not found
Oct 30 01:14:40.316639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.316462    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.316639 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:14:40.316545    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls podName:fb114e8e-a391-43ca-aaae-bf52d6518b37 nodeName:}" failed. No retries permitted until 2026-10-30 01:14:40.816523459 +0000 UTC m=+1988.845833684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls") pod "maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" (UID: "fb114e8e-a391-43ca-aaae-bf52d6518b37") : secret "maas-api-e2e-shared-a-w2-35042a-serving-cert" not found
Oct 30 01:14:40.317130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.317099    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.319214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.319193    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.421915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.421826    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f7zl\" (UniqueName: \"kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.539158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.539122    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:40.827378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.827340    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.830186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.830154    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:40.986086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:40.985578    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:41.305413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:41.305380    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"]
Oct 30 01:14:41.306043 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:41.306019    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb114e8e_a391_43ca_aaae_bf52d6518b37.slice/crio-595ebd0f1755fe07c19437295c07f4c55c49dff0d80a46315f7589fe11b5965a WatchSource:0}: Error finding container 595ebd0f1755fe07c19437295c07f4c55c49dff0d80a46315f7589fe11b5965a: Status 404 returned error can't find the container with id 595ebd0f1755fe07c19437295c07f4c55c49dff0d80a46315f7589fe11b5965a
Oct 30 01:14:42.148997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:42.148961    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" event={"ID":"fb114e8e-a391-43ca-aaae-bf52d6518b37","Type":"ContainerStarted","Data":"eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165"}
Oct 30 01:14:42.149198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:42.149024    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" event={"ID":"fb114e8e-a391-43ca-aaae-bf52d6518b37","Type":"ContainerStarted","Data":"595ebd0f1755fe07c19437295c07f4c55c49dff0d80a46315f7589fe11b5965a"}
Oct 30 01:14:42.149198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:42.149054    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:42.198446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:42.198395    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" podStartSLOduration=1.846965985 podStartE2EDuration="2.198382173s" totalImagesPullingTime="351.416188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:42.194909107 +0000 UTC m=+1990.224219338" watchObservedRunningTime="2026-10-30 01:14:42.198382173 +0000 UTC m=+1990.227692403"
Oct 30 01:14:42.202584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:42.202544    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:44.192414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:44.192162    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:44.552891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:44.546228    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:44.991879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:44.988987    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:45.572234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:45.552421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:45.880459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:45.861426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:46.145092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:46.144986    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:48.033889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:48.033805    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:48.618199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:48.618161    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:14:49.442034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:49.441994    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:49.682656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:49.673407    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:49.855903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:49.841657    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"]
Oct 30 01:14:49.855903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:49.841973    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" podUID="e171ba35-6cef-477e-a66a-090bc73715c9" containerName="maas-api" containerID="cri-o://82b715126cf0700500b18e4ea6bb46b47c506354a9c2da8fccc656f6b44ba0c8" gracePeriod=30
Oct 30 01:14:50.086218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.086155    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:50.265551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.265477    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:50.386178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.386096    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:50.512877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.511635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:50.942730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.942684    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:50.950864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.944905    4577 generic.go:356] "Generic (PLEG): container finished" podID="e171ba35-6cef-477e-a66a-090bc73715c9" containerID="82b715126cf0700500b18e4ea6bb46b47c506354a9c2da8fccc656f6b44ba0c8" exitCode=0
Oct 30 01:14:50.950864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:50.945069    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" event={"ID":"e171ba35-6cef-477e-a66a-090bc73715c9","Type":"ContainerDied","Data":"82b715126cf0700500b18e4ea6bb46b47c506354a9c2da8fccc656f6b44ba0c8"}
Oct 30 01:14:51.115873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.115078    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:14:51.144718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.144673    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:51.310554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.310455    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:51.513911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.447426    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\") pod \"e171ba35-6cef-477e-a66a-090bc73715c9\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") "
Oct 30 01:14:51.513911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.447615    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x\" (UniqueName: \"kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x\") pod \"e171ba35-6cef-477e-a66a-090bc73715c9\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") "
Oct 30 01:14:51.513911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.447690    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs\") pod \"e171ba35-6cef-477e-a66a-090bc73715c9\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") "
Oct 30 01:14:51.513911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.447773    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\") pod \"e171ba35-6cef-477e-a66a-090bc73715c9\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") "
Oct 30 01:14:51.578599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.573250    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca" pod "e171ba35-6cef-477e-a66a-090bc73715c9" (UID: "e171ba35-6cef-477e-a66a-090bc73715c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:51.621260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.621231    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\") pod \"e171ba35-6cef-477e-a66a-090bc73715c9\" (UID: \"e171ba35-6cef-477e-a66a-090bc73715c9\") "
Oct 30 01:14:51.636884 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:14:51.633008    4577 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/e171ba35-6cef-477e-a66a-090bc73715c9/volumes/kubernetes.io~configmap/openshift-service-ca
Oct 30 01:14:51.636884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.633041    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca" pod "e171ba35-6cef-477e-a66a-090bc73715c9" (UID: "e171ba35-6cef-477e-a66a-090bc73715c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:14:51.642904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.641396    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs" pod "e171ba35-6cef-477e-a66a-090bc73715c9" (UID: "e171ba35-6cef-477e-a66a-090bc73715c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:14:51.643519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.643482    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:51.649273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.643688    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e171ba35-6cef-477e-a66a-090bc73715c9-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:51.691455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.691385    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x" pod "e171ba35-6cef-477e-a66a-090bc73715c9" (UID: "e171ba35-6cef-477e-a66a-090bc73715c9"). InnerVolumeSpecName "kube-api-access-9fh4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:14:51.755784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.754575    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fh4x\" (UniqueName: \"kubernetes.io/projected/e171ba35-6cef-477e-a66a-090bc73715c9-kube-api-access-9fh4x\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:51.759287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.759240    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls" pod "e171ba35-6cef-477e-a66a-090bc73715c9" (UID: "e171ba35-6cef-477e-a66a-090bc73715c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:14:51.856205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.856090    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e171ba35-6cef-477e-a66a-090bc73715c9-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:14:51.992005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.991973    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg" event={"ID":"e171ba35-6cef-477e-a66a-090bc73715c9","Type":"ContainerDied","Data":"a6cba6491df6c98d3c630fb07097aab960cf88a127d541a6ae6ea5403b6c8705"}
Oct 30 01:14:51.992288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.992251    4577 scope.go:122] "RemoveContainer" containerID="82b715126cf0700500b18e4ea6bb46b47c506354a9c2da8fccc656f6b44ba0c8"
Oct 30 01:14:51.992685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:51.992667    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"
Oct 30 01:14:52.207928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:52.207756    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"]
Oct 30 01:14:52.207928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:52.207812    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-88a39d-5977bfc596-qpgsg"]
Oct 30 01:14:52.511204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:52.511164    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:54.146867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:54.142104    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e171ba35-6cef-477e-a66a-090bc73715c9" path="/var/lib/kubelet/pods/e171ba35-6cef-477e-a66a-090bc73715c9/volumes"
Oct 30 01:14:54.364871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:54.364790    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:55.304982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:55.304943    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:55.749949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:55.749918    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:56.059471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.059364    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:56.184528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.184494    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"]
Oct 30 01:14:56.185463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.185439    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e171ba35-6cef-477e-a66a-090bc73715c9" containerName="maas-api"
Oct 30 01:14:56.185649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.185631    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e171ba35-6cef-477e-a66a-090bc73715c9" containerName="maas-api"
Oct 30 01:14:56.186002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.185983    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e171ba35-6cef-477e-a66a-090bc73715c9" containerName="maas-api"
Oct 30 01:14:56.226877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.224929    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:56.250893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.249324    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"]
Oct 30 01:14:56.250893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.249482    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323185    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323321    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323350    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323473    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323524    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323573    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxv6w\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323664    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323768    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.323937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.323816    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.375374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.360244    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-c666c2-openshift-default-dockercfg-rsjx9\""
Oct 30 01:14:56.397879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.397344    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.425869    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426171    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426202    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426260    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426298    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426327    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxv6w\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426471    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426497    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.448935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.426526    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.458885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.453691    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.458885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.454104    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.458885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.454337    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.463878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.459489    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.463878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.460018    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.471160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.471094    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.477389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.477336    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.523874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.521421    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.612186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.544532    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:56.612186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.594662    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxv6w\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w\") pod \"e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") " pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:56.631549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:56.630597    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:57.080870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.080805    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:57.187082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.178150    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"]
Oct 30 01:14:57.281904 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:14:57.267226    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:14:57.294444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.293659    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:57.294444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.293749    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:57.294444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.293792    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:57.326302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.326273    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"]
Oct 30 01:14:57.326997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.326974    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.341571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.341496    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"]
Oct 30 01:14:57.349780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.349752    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-bdf6ca-openshift-default-dockercfg-2vvrg\""
Oct 30 01:14:57.386931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.386900    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.386946    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387055    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82zd6\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387133    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387173    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387224    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387375    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387497    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.387610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.387597    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.397947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.397914    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:57.490428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.490388    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.490795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.490768    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.491908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.491749    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.492199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.492178    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.492753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.492735    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.493365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.493167    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.493837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.493814    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82zd6\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.494046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.494027    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.494545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.494417    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.495107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.494964    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.495921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.495716    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.496098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.494494    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.496337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.496315    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.496907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.496733    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.497503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.497481    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.497656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.497634    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.527263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.526676    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82zd6\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.544366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.544288    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token\") pod \"e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") " pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.607463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.607350    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:57.687725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.687466    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:57.742725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.742630    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:57.973761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.973643    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:57.980960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.980797    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"]
Oct 30 01:14:57.983835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:57.981098    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" containerID="cri-o://b648baf58040354a8e6afc641b59fcbdbc1d42b9c4c03f40040145fd2911df61" gracePeriod=30
Oct 30 01:14:58.167743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.147365    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:14:58.292882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.288588    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" event={"ID":"fe335ad9-3734-4a58-a537-f9976621b4c6","Type":"ContainerStarted","Data":"96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"}
Oct 30 01:14:58.292882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.288670    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" event={"ID":"fe335ad9-3734-4a58-a537-f9976621b4c6","Type":"ContainerStarted","Data":"47544d75b2563c724167d9ece6870d3d8f1c5b476221a5bb276d0d34862aefb4"}
Oct 30 01:14:58.353374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.353299    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podStartSLOduration=2.353279641 podStartE2EDuration="2.353279641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:58.346565931 +0000 UTC m=+2006.375876198" watchObservedRunningTime="2026-10-30 01:14:58.353279641 +0000 UTC m=+2006.382589873"
Oct 30 01:14:58.397925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.397895    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"]
Oct 30 01:14:58.408575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.408538    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:58.408718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.408632    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:58.408718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.408679    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:14:58.631603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.631556    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:58.641907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:58.641827    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:59.249025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.248985    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:14:59.337828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.249055    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:14:59.337828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.300434    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" event={"ID":"92ddec96-952c-4a03-ae26-09aa2368fdfb","Type":"ContainerStarted","Data":"52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"}
Oct 30 01:14:59.337828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.300493    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" event={"ID":"92ddec96-952c-4a03-ae26-09aa2368fdfb","Type":"ContainerStarted","Data":"e4b368103419d10db1462511b4b02f33d66792bca849cded9f3e66e865cbdca9"}
Oct 30 01:14:59.337828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.300684    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:59.337828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.302110    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:14:59.352949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.352884    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podStartSLOduration=3.3528637 podStartE2EDuration="3.3528637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:59.349484909 +0000 UTC m=+2007.378795144" watchObservedRunningTime="2026-10-30 01:14:59.3528637 +0000 UTC m=+2007.382173928"
Oct 30 01:14:59.688467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.688428    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:14:59.695926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:14:59.695897    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:15:00.170137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.170106    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"]
Oct 30 01:15:00.173206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.173180    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": read tcp 10.134.0.2:37552->10.134.0.195:15021: read: connection reset by peer" start-of-body=
Oct 30 01:15:00.173347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.173237    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": read tcp 10.134.0.2:37552->10.134.0.195:15021: read: connection reset by peer"
Oct 30 01:15:00.174770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.174750    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.175996 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.175967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:00.248299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.248265    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"]
Oct 30 01:15:00.261968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.261939    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"]
Oct 30 01:15:00.265140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.262189    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.274908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.273112    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"]
Oct 30 01:15:00.277837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.277807    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.292917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.292798    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"]
Oct 30 01:15:00.296247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.295453    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:15:00.299732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.299667    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"]
Oct 30 01:15:00.315039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.313915    4577 generic.go:356] "Generic (PLEG): container finished" podID="c4b1183c-5416-461c-9375-7e377bb762a1" containerID="5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c" exitCode=0
Oct 30 01:15:00.315039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.314026    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" event={"ID":"c4b1183c-5416-461c-9375-7e377bb762a1","Type":"ContainerDied","Data":"5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"}
Oct 30 01:15:00.315039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.314472    4577 scope.go:122] "RemoveContainer" containerID="5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"
Oct 30 01:15:00.317982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.317883    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.318223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.318038    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.318223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.318086    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.341762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.341726    4577 generic.go:356] "Generic (PLEG): container finished" podID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerID="ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933" exitCode=0
Oct 30 01:15:00.341995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.341967    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrsg\" (UniqueName: \"kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.342087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.342062    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" event={"ID":"38c3a078-0da3-44e6-91b9-32604ce48f3d","Type":"ContainerDied","Data":"ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"}
Oct 30 01:15:00.342232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.342067    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888715-s2gfq\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.342445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.342424    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9jpm\" (UniqueName: \"kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm\") pod \"maas-api-key-cleanup-29888715-s2gfq\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.342533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.342518    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.342533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.342526    4577 scope.go:122] "RemoveContainer" containerID="ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"
Oct 30 01:15:00.346706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.346668    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.347009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.346985    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.347265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.347242    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.362908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.357338    4577 generic.go:356] "Generic (PLEG): container finished" podID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerID="a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472" exitCode=0
Oct 30 01:15:00.362908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.357877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" event={"ID":"9379b6ab-f35d-4d33-bca6-9c9073c290f9","Type":"ContainerDied","Data":"a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"}
Oct 30 01:15:00.362908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.358247    4577 scope.go:122] "RemoveContainer" containerID="a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"
Oct 30 01:15:00.362908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.358744    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:15:00.362908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.361213    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:15:00.366367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.365816    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.368072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.367971    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.368072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.368049    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:00.443705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.443673    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgd98\" (UniqueName: \"kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.443913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.443880    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.444403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.444375    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrsg\" (UniqueName: \"kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.444516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.444477    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.444651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.444629    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888715-s2gfq\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.444907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.444879    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9jpm\" (UniqueName: \"kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm\") pod \"maas-api-key-cleanup-29888715-s2gfq\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.445456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.445433    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.447307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.447281    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888715-s2gfq\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.475163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.475134    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrsg\" (UniqueName: \"kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.475471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.475432    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9jpm\" (UniqueName: \"kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm\") pod \"maas-api-key-cleanup-29888715-s2gfq\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.553870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.550486    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.553870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.550575    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgd98\" (UniqueName: \"kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.572880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.570384    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.607032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.606785    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:00.607859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.607772    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgd98\" (UniqueName: \"kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.617791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.617373    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:00.649759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.649516    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:00.983355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:00.974729    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:01.093497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:01.093312    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:01.181739 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:01.179150    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380dbe32_6f86_4cd5_8211_6b1621283bec.slice/crio-14f97b1b7641469fa053ca17e4ab079a1bc10d4be0df324b7c9a5117e1548758 WatchSource:0}: Error finding container 14f97b1b7641469fa053ca17e4ab079a1bc10d4be0df324b7c9a5117e1548758: Status 404 returned error can't find the container with id 14f97b1b7641469fa053ca17e4ab079a1bc10d4be0df324b7c9a5117e1548758
Oct 30 01:15:01.204243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:01.191610    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"]
Oct 30 01:15:01.230869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:01.224156    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"]
Oct 30 01:15:01.329702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:01.329193    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"]
Oct 30 01:15:01.506725 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:01.506689    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode928fbec_b0ae_4f9d_ba43_d8d90a6682ba.slice/crio-113977a2fc7057f4047c86e6977a8783ce7f9c5dd4861a5bde69b86236fb742f WatchSource:0}: Error finding container 113977a2fc7057f4047c86e6977a8783ce7f9c5dd4861a5bde69b86236fb742f: Status 404 returned error can't find the container with id 113977a2fc7057f4047c86e6977a8783ce7f9c5dd4861a5bde69b86236fb742f
Oct 30 01:15:01.614001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:01.613937    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" event={"ID":"9379b6ab-f35d-4d33-bca6-9c9073c290f9","Type":"ContainerStarted","Data":"ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de"}
Oct 30 01:15:01.729870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:01.726562    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" event={"ID":"f5e05e57-8619-4454-96b7-671b7d811e31","Type":"ContainerStarted","Data":"71e9ab2307b8fab70f777e0a344e5ba3ccfe4928c422964da8cda54afa39cf8d"}
Oct 30 01:15:02.085334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.085012    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" event={"ID":"c4b1183c-5416-461c-9375-7e377bb762a1","Type":"ContainerStarted","Data":"def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777"}
Oct 30 01:15:02.209910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.121559    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:02.351923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.247169    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:02.351923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.247225    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:02.471374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.471008    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:02.471374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.471087    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:02.474719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.474204    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:15:02.474719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.474264    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:15:02.474719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.474291    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"]
Oct 30 01:15:02.474719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.474330    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"]
Oct 30 01:15:02.495748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.495712    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy" containerID="cri-o://bf5b1cbcd3e4ad4e9360ec4c14c56595db2e19aed7a412b417e6d91da416abf5" gracePeriod=30
Oct 30 01:15:02.516091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.511097    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"]
Oct 30 01:15:02.516091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.511272    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.519534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.519503    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" event={"ID":"38c3a078-0da3-44e6-91b9-32604ce48f3d","Type":"ContainerStarted","Data":"38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4"}
Oct 30 01:15:02.522254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.522224    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:02.557870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.555144    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 01:15:02.572884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.565015    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" event={"ID":"380dbe32-6f86-4cd5-8211-6b1621283bec","Type":"ContainerStarted","Data":"14f97b1b7641469fa053ca17e4ab079a1bc10d4be0df324b7c9a5117e1548758"}
Oct 30 01:15:02.634906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.634226    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:15:02.652457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.650463    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:15:02.652457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.651553    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:02.652457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.651600    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:02.737015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.729696    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.737015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.729773    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckmmg\" (UniqueName: \"kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.737015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.729809    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.896924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.887583    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" podStartSLOduration=2.8875591590000003 podStartE2EDuration="2.887559159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:02.802479428 +0000 UTC m=+2010.831789660" watchObservedRunningTime="2026-10-30 01:15:02.887559159 +0000 UTC m=+2010.916869390"
Oct 30 01:15:02.896924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.893031    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.896924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.893081    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckmmg\" (UniqueName: \"kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.896924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.893117    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.896924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.893947    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.921212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.920337    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:02.952945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:02.946114    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckmmg\" (UniqueName: \"kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg\") pod \"maas-api-revoke-sub-5e54b5f8-vqs4g\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:03.065788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.058680    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:03.231944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.231287    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:03.231944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.231340    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:03.441184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.440701    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:03.441184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.440754    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:03.611064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.607965    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:03.611064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.608025    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:03.810402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.797121    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerStarted","Data":"be7341e79b15c778f82598346a454f7002c108f800c7ad0cac76ecffa54d46fd"}
Oct 30 01:15:03.810402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:03.797188    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerStarted","Data":"113977a2fc7057f4047c86e6977a8783ce7f9c5dd4861a5bde69b86236fb742f"}
Oct 30 01:15:04.102924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.099435    4577 generic.go:356] "Generic (PLEG): container finished" podID="f5e05e57-8619-4454-96b7-671b7d811e31" containerID="c53d9c32dccb50d6d464c5b0553b2bc7356249b31d6d756f6b296e7c23d657df" exitCode=60
Oct 30 01:15:04.102924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.099638    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" event={"ID":"f5e05e57-8619-4454-96b7-671b7d811e31","Type":"ContainerDied","Data":"c53d9c32dccb50d6d464c5b0553b2bc7356249b31d6d756f6b296e7c23d657df"}
Oct 30 01:15:04.102924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.100168    4577 scope.go:122] "RemoveContainer" containerID="c53d9c32dccb50d6d464c5b0553b2bc7356249b31d6d756f6b296e7c23d657df"
Oct 30 01:15:04.179775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.178668    4577 patch_prober.go:29] interesting pod/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused" start-of-body=
Oct 30 01:15:04.179775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.178725    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.181:15021/healthz/ready\": dial tcp 10.134.0.181:15021: connect: connection refused"
Oct 30 01:15:04.342978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.340986    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:04.342978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.341035    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:04.342978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.341644    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:04.348869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.344872    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" podStartSLOduration=4.344829887 podStartE2EDuration="4.344829887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:03.906126879 +0000 UTC m=+2011.935437111" watchObservedRunningTime="2026-10-30 01:15:04.344829887 +0000 UTC m=+2012.374140119"
Oct 30 01:15:04.442745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.442655    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:04.442745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.442733    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:04.538628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.538583    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"]
Oct 30 01:15:04.541429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.541394    4577 generic.go:356] "Generic (PLEG): container finished" podID="0464e965-6efa-49ae-a42d-900029d35dbc" containerID="b648baf58040354a8e6afc641b59fcbdbc1d42b9c4c03f40040145fd2911df61" exitCode=0
Oct 30 01:15:04.541633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.541615    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" event={"ID":"0464e965-6efa-49ae-a42d-900029d35dbc","Type":"ContainerDied","Data":"b648baf58040354a8e6afc641b59fcbdbc1d42b9c4c03f40040145fd2911df61"}
Oct 30 01:15:04.541687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.541671    4577 scope.go:122] "RemoveContainer" containerID="99168b2a20b3f8d1db384c023678394e70b301cb65b85bf1d734a4d532fd0e10"
Oct 30 01:15:04.542141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.542120    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:04.628115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.607582    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:04.628115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.607642    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:04.861173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:04.861061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:05.313987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.313924    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:05.314364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.314332    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:05.315565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.315539    4577 generic.go:356] "Generic (PLEG): container finished" podID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerID="7f706dbfe5398c5976bf450382481cf3d4d7bdce5f9b1ee0e731e33ca6cfaa7f" exitCode=60
Oct 30 01:15:05.347215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.317779    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" event={"ID":"380dbe32-6f86-4cd5-8211-6b1621283bec","Type":"ContainerDied","Data":"7f706dbfe5398c5976bf450382481cf3d4d7bdce5f9b1ee0e731e33ca6cfaa7f"}
Oct 30 01:15:05.690710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.690658    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:05.691138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.691099    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:05.714083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.690912    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:05.714083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.700304    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:05.744940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.714448    4577 scope.go:122] "RemoveContainer" containerID="7f706dbfe5398c5976bf450382481cf3d4d7bdce5f9b1ee0e731e33ca6cfaa7f"
Oct 30 01:15:05.942023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.788674    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:15:06.003862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:05.997515    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:06.049490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.049429    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:15:06.364926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.335165    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:06.364926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.335240    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:06.515882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.502616    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:15:06.818909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.556308    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:15:06.818909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.563222    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:06.818909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.807283    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:06.818909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.651143    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:06.818909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.807333    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:06.818909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.667820    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:06.839362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.832430    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"]
Oct 30 01:15:06.839362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.832691    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" containerID="cri-o://e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff" gracePeriod=30
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.933792    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.933888    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.933943    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.934060    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.934089    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.934195    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.934347    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.934397    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:06.939205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:06.934429    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs\") pod \"0464e965-6efa-49ae-a42d-900029d35dbc\" (UID: \"0464e965-6efa-49ae-a42d-900029d35dbc\") "
Oct 30 01:15:07.037023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.036975    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:07.048298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.048185    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:07.048529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.048455    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:07.101884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.050289    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:07.101884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.060645    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.101884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.060677    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464e965-6efa-49ae-a42d-900029d35dbc-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.101884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.060695    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.101884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.060708    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.119881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.112902    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:07.130170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.120617    4577 generic.go:356] "Generic (PLEG): container finished" podID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerID="be7341e79b15c778f82598346a454f7002c108f800c7ad0cac76ecffa54d46fd" exitCode=60
Oct 30 01:15:07.130170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.120733    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerDied","Data":"be7341e79b15c778f82598346a454f7002c108f800c7ad0cac76ecffa54d46fd"}
Oct 30 01:15:07.130170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.121339    4577 scope.go:122] "RemoveContainer" containerID="be7341e79b15c778f82598346a454f7002c108f800c7ad0cac76ecffa54d46fd"
Oct 30 01:15:07.168878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.164561    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.229884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.229819    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "kube-api-access-h9k58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:07.230770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.230719    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" event={"ID":"f5e05e57-8619-4454-96b7-671b7d811e31","Type":"ContainerStarted","Data":"e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba"}
Oct 30 01:15:07.231916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.231822    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:07.233521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.233496    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:07.274884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.271222    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9k58\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-kube-api-access-h9k58\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.355994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.345112    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g" event={"ID":"64fe5aba-5ede-4535-9991-4285a8038904","Type":"ContainerStarted","Data":"3a277634ab3a3bc9e938e83bc4e1e6a180015e9f61c9832e09eafdc213e003f9"}
Oct 30 01:15:07.355994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.345179    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g" event={"ID":"64fe5aba-5ede-4535-9991-4285a8038904","Type":"ContainerStarted","Data":"9e5d38a56135a7950859f2a16dda784a23c68495ef457fdaf8bcda24323a5c2e"}
Oct 30 01:15:07.418358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.413996    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:15:07.555665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.544208    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:07.565659 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.565590    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:07.566065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.566001    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:07.566576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.566488    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token" pod "0464e965-6efa-49ae-a42d-900029d35dbc" (UID: "0464e965-6efa-49ae-a42d-900029d35dbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:07.655237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.640319    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"
Oct 30 01:15:07.655237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.640520    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v" event={"ID":"0464e965-6efa-49ae-a42d-900029d35dbc","Type":"ContainerDied","Data":"44d5b90e8e2a502917a6df9dd9bc7a6af90c407fa3fd4ab3f3af4f8c0e5d9b2f"}
Oct 30 01:15:07.655237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.640589    4577 scope.go:122] "RemoveContainer" containerID="b648baf58040354a8e6afc641b59fcbdbc1d42b9c4c03f40040145fd2911df61"
Oct 30 01:15:07.666032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.665997    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:07.729903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.694293    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:07.729903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.678339    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464e965-6efa-49ae-a42d-900029d35dbc-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.729903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.694387    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464e965-6efa-49ae-a42d-900029d35dbc-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.729903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.694402    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464e965-6efa-49ae-a42d-900029d35dbc-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:07.729903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.728378    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" podStartSLOduration=7.728358006 podStartE2EDuration="7.728358006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:07.400002401 +0000 UTC m=+2015.429312637" watchObservedRunningTime="2026-10-30 01:15:07.728358006 +0000 UTC m=+2015.757668259"
Oct 30 01:15:07.741181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.737204    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:07.769128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:07.769053    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g" podStartSLOduration=6.769013332 podStartE2EDuration="6.769013332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:07.691520748 +0000 UTC m=+2015.720830981" watchObservedRunningTime="2026-10-30 01:15:07.769013332 +0000 UTC m=+2015.798323563"
Oct 30 01:15:08.010016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.008899    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:08.012686 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.012660    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"]
Oct 30 01:15:08.093028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.092987    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:15:08.208872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.200258    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-88a39d-openshift-default-784fdc8cd5-m6f6v"]
Oct 30 01:15:08.232305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.232269    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused" start-of-body=
Oct 30 01:15:08.232641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.232609    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.195:15021/healthz/ready\": dial tcp 10.134.0.195:15021: connect: connection refused"
Oct 30 01:15:08.315099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.314823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:08.417287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.416723    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused" start-of-body=
Oct 30 01:15:08.417287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.416783    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.192:15021/healthz/ready\": dial tcp 10.134.0.192:15021: connect: connection refused"
Oct 30 01:15:08.517965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.515441    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.627920    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"]
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.629015    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy"
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.629038    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy"
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.629086    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy"
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.629094    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy"
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.629252    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy"
Oct 30 01:15:08.668934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.629271    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" containerName="istio-proxy"
Oct 30 01:15:08.678933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.678882    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused" start-of-body=
Oct 30 01:15:08.679268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.679214    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.194:15021/healthz/ready\": dial tcp 10.134.0.194:15021: connect: connection refused"
Oct 30 01:15:08.969881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:08.965907    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.063222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.063190    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-9da49f-openshift-default-dockercfg-9sqgl\""
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.075593    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"]
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.077787    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.105212    4577 generic.go:356] "Generic (PLEG): container finished" podID="f5e05e57-8619-4454-96b7-671b7d811e31" containerID="e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba" exitCode=60
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.105374    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" event={"ID":"f5e05e57-8619-4454-96b7-671b7d811e31","Type":"ContainerDied","Data":"e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba"}
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.105431    4577 scope.go:122] "RemoveContainer" containerID="c53d9c32dccb50d6d464c5b0553b2bc7356249b31d6d756f6b296e7c23d657df"
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.114102    4577 generic.go:356] "Generic (PLEG): container finished" podID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerID="bf5b1cbcd3e4ad4e9360ec4c14c56595db2e19aed7a412b417e6d91da416abf5" exitCode=0
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.114287    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" event={"ID":"9f3c8e5c-7027-4961-b0bb-0705746ab4c8","Type":"ContainerDied","Data":"bf5b1cbcd3e4ad4e9360ec4c14c56595db2e19aed7a412b417e6d91da416abf5"}
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.141885    4577 generic.go:356] "Generic (PLEG): container finished" podID="64fe5aba-5ede-4535-9991-4285a8038904" containerID="3a277634ab3a3bc9e938e83bc4e1e6a180015e9f61c9832e09eafdc213e003f9" exitCode=0
Oct 30 01:15:09.166885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.142016    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g" event={"ID":"64fe5aba-5ede-4535-9991-4285a8038904","Type":"ContainerDied","Data":"3a277634ab3a3bc9e938e83bc4e1e6a180015e9f61c9832e09eafdc213e003f9"}
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.170717    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.170825    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.170943    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.171067    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.171197    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.171271    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8m82\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.171436    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.171485    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.172219    4577 scope.go:122] "RemoveContainer" containerID="e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba"
Oct 30 01:15:09.188122 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:09.172580    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf_odh-ai-gateway-infra(f5e05e57-8619-4454-96b7-671b7d811e31)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" podUID="f5e05e57-8619-4454-96b7-671b7d811e31"
Oct 30 01:15:09.192295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.192227    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" event={"ID":"380dbe32-6f86-4cd5-8211-6b1621283bec","Type":"ContainerStarted","Data":"c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"}
Oct 30 01:15:09.193244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.193222    4577 scope.go:122] "RemoveContainer" containerID="c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"
Oct 30 01:15:09.196468 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:09.196441    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9_odh-ai-gateway-infra(380dbe32-6f86-4cd5-8211-6b1621283bec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec"
Oct 30 01:15:09.198480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.198458    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:15:09.285536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.285497    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.285775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.285758    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.285921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.285906    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.286167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.286149    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.286320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.286306    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.286554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.286537    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.286973 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.286949    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.287176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.287162    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.287379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.287366    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.287547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.287534    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.287698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.287683    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8m82\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.287919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.287904    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.288115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.288099    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.288325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.288312    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.288491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.288479    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.289279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.289256    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.290354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.290332    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.291176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.290506    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:09.291545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.290820    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:09.292180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.292160    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.293365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.293344    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.294107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.294086    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.299366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.299337    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:09.303864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.301959    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:09.376228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.374752    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:15:09.376557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.376534    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:15:09.379247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.379151    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "kube-api-access-ngmwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:09.379386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.379325    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.387933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.387192    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:09.387933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.387703    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.388428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.388235    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.389607    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.389648    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.389736    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data\") pod \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\" (UID: \"9f3c8e5c-7027-4961-b0bb-0705746ab4c8\") "
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390468    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390488    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390501    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390514    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390527    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.390623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390540    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngmwm\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-kube-api-access-ngmwm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.391193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.390745    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:09.396776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.393369    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:15:09.396776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.395989    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token" pod "9f3c8e5c-7027-4961-b0bb-0705746ab4c8" (UID: "9f3c8e5c-7027-4961-b0bb-0705746ab4c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:09.400049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.399833    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:15:09.401166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.400316    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8m82\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82\") pod \"e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") " pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.440288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.440224    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:09.548212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.548170    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:09.557196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.555438    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.557196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.555469    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.557196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.555481    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f3c8e5c-7027-4961-b0bb-0705746ab4c8-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:09.667631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.661505    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:15:09.668189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.668157    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:15:09.669133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.668747    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:15:09.672702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.672653    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:15:09.674665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.674602    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:15:09.921001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.920970    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"]
Oct 30 01:15:09.921369 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:09.921345    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9998a2ee_6eba_4438_8713_af86376d2dc2.slice/crio-3cffeb19c3c052b9f89b324f311dd1d9fed51e7536773212c48929f01d3479d2 WatchSource:0}: Error finding container 3cffeb19c3c052b9f89b324f311dd1d9fed51e7536773212c48929f01d3479d2: Status 404 returned error can't find the container with id 3cffeb19c3c052b9f89b324f311dd1d9fed51e7536773212c48929f01d3479d2
Oct 30 01:15:09.924864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.924800    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:09.925002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.924907    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:09.925002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:09.924952    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:10.073490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.073452    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"]
Oct 30 01:15:10.074229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.074211    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy"
Oct 30 01:15:10.074288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.074232    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy"
Oct 30 01:15:10.074345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.074340    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" containerName="istio-proxy"
Oct 30 01:15:10.077067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.077051    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.117867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.115091    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-1854c4-serving-cert\""
Oct 30 01:15:10.127410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.127365    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0464e965-6efa-49ae-a42d-900029d35dbc" path="/var/lib/kubelet/pods/0464e965-6efa-49ae-a42d-900029d35dbc/volumes"
Oct 30 01:15:10.128338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.128320    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"]
Oct 30 01:15:10.167694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.167647    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.167940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.167719    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhc4v\" (UniqueName: \"kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.168547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.168446    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.169353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.169326    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.171611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.171586    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:10.205449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.205419    4577 generic.go:356] "Generic (PLEG): container finished" podID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerID="f0123cb48dc102455847692df2ea24250b2fa86fca467d2760f941679eb22c95" exitCode=60
Oct 30 01:15:10.205627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.205508    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerDied","Data":"f0123cb48dc102455847692df2ea24250b2fa86fca467d2760f941679eb22c95"}
Oct 30 01:15:10.205627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.205557    4577 scope.go:122] "RemoveContainer" containerID="be7341e79b15c778f82598346a454f7002c108f800c7ad0cac76ecffa54d46fd"
Oct 30 01:15:10.205966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.205936    4577 scope.go:122] "RemoveContainer" containerID="f0123cb48dc102455847692df2ea24250b2fa86fca467d2760f941679eb22c95"
Oct 30 01:15:10.206382 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:10.206345    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888715-s2gfq_odh-ai-gateway-infra(e928fbec-b0ae-4f9d-ba43-d8d90a6682ba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba"
Oct 30 01:15:10.207697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.207675    4577 scope.go:122] "RemoveContainer" containerID="e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba"
Oct 30 01:15:10.208057 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:10.208033    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf_odh-ai-gateway-infra(f5e05e57-8619-4454-96b7-671b7d811e31)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" podUID="f5e05e57-8619-4454-96b7-671b7d811e31"
Oct 30 01:15:10.209212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.209193    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"
Oct 30 01:15:10.209330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.209222    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn" event={"ID":"9f3c8e5c-7027-4961-b0bb-0705746ab4c8","Type":"ContainerDied","Data":"a9a6f642fe5a83d0c8de108d65b1446ebf29ddea24108ba60ce1e66313705d18"}
Oct 30 01:15:10.212410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.212386    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" event={"ID":"9998a2ee-6eba-4438-8713-af86376d2dc2","Type":"ContainerStarted","Data":"bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"}
Oct 30 01:15:10.212510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.212425    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" event={"ID":"9998a2ee-6eba-4438-8713-af86376d2dc2","Type":"ContainerStarted","Data":"3cffeb19c3c052b9f89b324f311dd1d9fed51e7536773212c48929f01d3479d2"}
Oct 30 01:15:10.215174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.215151    4577 generic.go:356] "Generic (PLEG): container finished" podID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerID="c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba" exitCode=60
Oct 30 01:15:10.215282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.215226    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" event={"ID":"380dbe32-6f86-4cd5-8211-6b1621283bec","Type":"ContainerDied","Data":"c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"}
Oct 30 01:15:10.219204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.219183    4577 scope.go:122] "RemoveContainer" containerID="c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"
Oct 30 01:15:10.219425 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:10.219409    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9_odh-ai-gateway-infra(380dbe32-6f86-4cd5-8211-6b1621283bec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec"
Oct 30 01:15:10.222594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.222576    4577 scope.go:122] "RemoveContainer" containerID="bf5b1cbcd3e4ad4e9360ec4c14c56595db2e19aed7a412b417e6d91da416abf5"
Oct 30 01:15:10.235122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.235095    4577 scope.go:122] "RemoveContainer" containerID="7f706dbfe5398c5976bf450382481cf3d4d7bdce5f9b1ee0e731e33ca6cfaa7f"
Oct 30 01:15:10.272031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.271998    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.272453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.272382    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.272731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.272713    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.272940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.272924    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhc4v\" (UniqueName: \"kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.274087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.273998    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.277582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.277546    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.277696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.277673    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.289129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.289099    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:10.324123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.323967    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhc4v\" (UniqueName: \"kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v\") pod \"maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.398717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.398677    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:10.452820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.452728    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"]
Oct 30 01:15:10.463624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.463229    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-lqpbn"]
Oct 30 01:15:10.509960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.509830    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:10.522591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.519011    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podStartSLOduration=2.518997971 podStartE2EDuration="2.518997971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:10.518537278 +0000 UTC m=+2018.547847512" watchObservedRunningTime="2026-10-30 01:15:10.518997971 +0000 UTC m=+2018.548308201"
Oct 30 01:15:10.555014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.554984    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:10.600469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.600439    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca\") pod \"64fe5aba-5ede-4535-9991-4285a8038904\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") "
Oct 30 01:15:10.600639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.600491    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg\" (UniqueName: \"kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg\") pod \"64fe5aba-5ede-4535-9991-4285a8038904\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") "
Oct 30 01:15:10.600639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.600582    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca\") pod \"64fe5aba-5ede-4535-9991-4285a8038904\" (UID: \"64fe5aba-5ede-4535-9991-4285a8038904\") "
Oct 30 01:15:10.601297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.601272    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca" pod "64fe5aba-5ede-4535-9991-4285a8038904" (UID: "64fe5aba-5ede-4535-9991-4285a8038904"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:10.603391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.603366    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca" pod "64fe5aba-5ede-4535-9991-4285a8038904" (UID: "64fe5aba-5ede-4535-9991-4285a8038904"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:15:10.603534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.603440    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg" pod "64fe5aba-5ede-4535-9991-4285a8038904" (UID: "64fe5aba-5ede-4535-9991-4285a8038904"). InnerVolumeSpecName "kube-api-access-ckmmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:10.719959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.702675    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:10.719959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.717967    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckmmg\" (UniqueName: \"kubernetes.io/projected/64fe5aba-5ede-4535-9991-4285a8038904-kube-api-access-ckmmg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:10.719959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.717998    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64fe5aba-5ede-4535-9991-4285a8038904-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:10.810500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:10.810469    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"]
Oct 30 01:15:10.810668 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:10.810477    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54abb8e7_cc15_4828_970b_3b0493136625.slice/crio-52d2025c3e3012afb10a3f0adc87559dc8eecbca394bec7563062b99f1dd0408 WatchSource:0}: Error finding container 52d2025c3e3012afb10a3f0adc87559dc8eecbca394bec7563062b99f1dd0408: Status 404 returned error can't find the container with id 52d2025c3e3012afb10a3f0adc87559dc8eecbca394bec7563062b99f1dd0408
Oct 30 01:15:11.224096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.224062    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz" event={"ID":"54abb8e7-cc15-4828-970b-3b0493136625","Type":"ContainerStarted","Data":"52d2025c3e3012afb10a3f0adc87559dc8eecbca394bec7563062b99f1dd0408"}
Oct 30 01:15:11.226340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.226315    4577 scope.go:122] "RemoveContainer" containerID="f0123cb48dc102455847692df2ea24250b2fa86fca467d2760f941679eb22c95"
Oct 30 01:15:11.226671 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:11.226646    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888715-s2gfq_odh-ai-gateway-infra(e928fbec-b0ae-4f9d-ba43-d8d90a6682ba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba"
Oct 30 01:15:11.228749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.228710    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"
Oct 30 01:15:11.228878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.228757    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g" event={"ID":"64fe5aba-5ede-4535-9991-4285a8038904","Type":"ContainerDied","Data":"9e5d38a56135a7950859f2a16dda784a23c68495ef457fdaf8bcda24323a5c2e"}
Oct 30 01:15:11.228878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.228815    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e5d38a56135a7950859f2a16dda784a23c68495ef457fdaf8bcda24323a5c2e"
Oct 30 01:15:11.231100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.231080    4577 scope.go:122] "RemoveContainer" containerID="c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"
Oct 30 01:15:11.231531 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:11.231508    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9_odh-ai-gateway-infra(380dbe32-6f86-4cd5-8211-6b1621283bec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec"
Oct 30 01:15:11.561305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:11.561271    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:12.115527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.115496    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f3c8e5c-7027-4961-b0bb-0705746ab4c8" path="/var/lib/kubelet/pods/9f3c8e5c-7027-4961-b0bb-0705746ab4c8/volumes"
Oct 30 01:15:12.194332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.194274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:12.206099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.206074    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.298523    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.298606    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.298683    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.298729    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.298910    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.298958    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.299002    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.299044    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.302064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.299073    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data\") pod \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\" (UID: \"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4\") "
Oct 30 01:15:12.312688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.311794    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:12.322878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.322189    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:12.322878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.322509    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:12.323143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.323097    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:12.328501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.328468    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:15:12.340904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.340297    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:12.340904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.340785    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:12.341111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.341059    4577 generic.go:356] "Generic (PLEG): container finished" podID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerID="e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff" exitCode=0
Oct 30 01:15:12.343753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.341276    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"
Oct 30 01:15:12.343753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.341375    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" event={"ID":"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4","Type":"ContainerDied","Data":"e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff"}
Oct 30 01:15:12.343753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.341496    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" event={"ID":"93ff1993-06bf-48b1-b9a0-05dc8b9b1be4","Type":"ContainerDied","Data":"ef37d9b6015d85ba3283c270cce8e57d902efbf76837ee8d5ab5e9bdd98bffff"}
Oct 30 01:15:12.343753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.341543    4577 scope.go:122] "RemoveContainer" containerID="e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff"
Oct 30 01:15:12.348692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.346886    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "kube-api-access-xsbj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:12.352600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.352475    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token" pod "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" (UID: "93ff1993-06bf-48b1-b9a0-05dc8b9b1be4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:12.355898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.353556    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz" event={"ID":"54abb8e7-cc15-4828-970b-3b0493136625","Type":"ContainerStarted","Data":"b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7"}
Oct 30 01:15:12.355898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.353915    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:12.358674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.358650    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:12.370941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.370913    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:15:12.399523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.398922    4577 scope.go:122] "RemoveContainer" containerID="e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff"
Oct 30 01:15:12.399523 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:12.399371    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff\": container with ID starting with e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff not found: ID does not exist" containerID="e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff"
Oct 30 01:15:12.399523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.399438    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff"} err="failed to get container status \"e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff\": rpc error: code = NotFound desc = could not find container \"e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff\": container with ID starting with e29ce2b6a5ac0fc7b9bee5ffd50a37e49942c91756d3bf23385603c3f28e98ff not found: ID does not exist"
Oct 30 01:15:12.401952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.401924    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.401957    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.401972    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsbj4\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-kube-api-access-xsbj4\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.401987    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.402000    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.402013    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.402025    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.402037    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.402072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.402048    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:12.414460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.414417    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz" podStartSLOduration=2.07494323 podStartE2EDuration="2.414404066s" totalImagesPullingTime="339.460836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:12.413278464 +0000 UTC m=+2020.442588696" watchObservedRunningTime="2026-10-30 01:15:12.414404066 +0000 UTC m=+2020.443714296"
Oct 30 01:15:12.706657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.706623    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"]
Oct 30 01:15:12.716190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.716165    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx"]
Oct 30 01:15:12.924087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:12.924013    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:13.044522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:13.044489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:13.172091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:13.172050    4577 patch_prober.go:29] interesting pod/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:15:13.172265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:13.172120    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-72ed13-openshift-default-74c5595989-cc8hx" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:15:14.115775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.115739    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" path="/var/lib/kubelet/pods/93ff1993-06bf-48b1-b9a0-05dc8b9b1be4/volumes"
Oct 30 01:15:14.318610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.318572    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:14.715605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.715577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:14.833260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.833211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:14.894568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.894534    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"]
Oct 30 01:15:14.895404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.895368    4577 cpu_manager.go:414] "containerMap: removing container" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy"
Oct 30 01:15:14.895404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.895398    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy"
Oct 30 01:15:14.895627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.895424    4577 cpu_manager.go:414] "containerMap: removing container" podUID="64fe5aba-5ede-4535-9991-4285a8038904" containerName="revoke-keys"
Oct 30 01:15:14.895627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.895432    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64fe5aba-5ede-4535-9991-4285a8038904" containerName="revoke-keys"
Oct 30 01:15:14.895733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.895645    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="93ff1993-06bf-48b1-b9a0-05dc8b9b1be4" containerName="istio-proxy"
Oct 30 01:15:14.895733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.895663    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="64fe5aba-5ede-4535-9991-4285a8038904" containerName="revoke-keys"
Oct 30 01:15:14.899762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.899740    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:14.923539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.923509    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"]
Oct 30 01:15:14.932214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.932185    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-9da49f-dockercfg-sl5tm\""
Oct 30 01:15:15.000942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:14.975117    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-9da49f\""
Oct 30 01:15:15.143439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.143401    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.146971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.146940    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdd7\" (UniqueName: \"kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7\") pod \"payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.250246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.250205    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdd7\" (UniqueName: \"kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7\") pod \"payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.250406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.250343    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.251390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.251340    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.277053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.277021    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdd7\" (UniqueName: \"kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7\") pod \"payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.520391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.520306    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:15.864237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:15.864146    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"]
Oct 30 01:15:16.294136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.294096    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:16.426267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.425603    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" event={"ID":"9a423130-f279-40f4-86fb-6120109a2a0b","Type":"ContainerStarted","Data":"6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac"}
Oct 30 01:15:16.426267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.425682    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" event={"ID":"9a423130-f279-40f4-86fb-6120109a2a0b","Type":"ContainerStarted","Data":"6272c776e8e27bafa1280a50a200e8ab1079e428ebb17f00a95cd2ebda27682f"}
Oct 30 01:15:16.427217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.427032    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:16.477495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.477443    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" podStartSLOduration=2.477428036 podStartE2EDuration="2.477428036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:16.47426223 +0000 UTC m=+2024.503572466" watchObservedRunningTime="2026-10-30 01:15:16.477428036 +0000 UTC m=+2024.506738274"
Oct 30 01:15:16.571117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.571039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:16.683231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:16.683192    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:18.122076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:18.122038    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:18.370631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:18.370603    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:15:19.879472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:19.879429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:20.104343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:20.104314    4577 scope.go:122] "RemoveContainer" containerID="e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba"
Oct 30 01:15:21.103700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.103671    4577 scope.go:122] "RemoveContainer" containerID="f0123cb48dc102455847692df2ea24250b2fa86fca467d2760f941679eb22c95"
Oct 30 01:15:21.473569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.473535    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerStarted","Data":"aed52d0e64a796ee12fcb42250831b7e79e5c6cf5aadd2bf8250c2ccea61b384"}
Oct 30 01:15:21.476046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.476017    4577 generic.go:356] "Generic (PLEG): container finished" podID="f5e05e57-8619-4454-96b7-671b7d811e31" containerID="a8fc8befd93b7d7fdae657e942b8fdac6cc776cae7b5f0e03a6eb703c877a89d" exitCode=60
Oct 30 01:15:21.476155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.476089    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" event={"ID":"f5e05e57-8619-4454-96b7-671b7d811e31","Type":"ContainerDied","Data":"a8fc8befd93b7d7fdae657e942b8fdac6cc776cae7b5f0e03a6eb703c877a89d"}
Oct 30 01:15:21.476200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.476154    4577 scope.go:122] "RemoveContainer" containerID="e3bc0f0bd79a5dd19641a1ba6ccee70fc5d0f2459d5813afe615e33cb210cfba"
Oct 30 01:15:21.476613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.476594    4577 scope.go:122] "RemoveContainer" containerID="a8fc8befd93b7d7fdae657e942b8fdac6cc776cae7b5f0e03a6eb703c877a89d"
Oct 30 01:15:21.477013 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:21.476991    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf_odh-ai-gateway-infra(f5e05e57-8619-4454-96b7-671b7d811e31)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" podUID="f5e05e57-8619-4454-96b7-671b7d811e31"
Oct 30 01:15:21.567788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.567750    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"]
Oct 30 01:15:21.573065 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.573041    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.590071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.590047    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-c666c2-serving-cert\""
Oct 30 01:15:21.594268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.594240    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"]
Oct 30 01:15:21.612925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.605737    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.612925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.605832    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.612925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.605986    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.612925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.606099    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvbf4\" (UniqueName: \"kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.669298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.669264    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:21.708160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.708129    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvbf4\" (UniqueName: \"kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.708325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.708277    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.708325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.708320    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.708406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.708359    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.709088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.709064    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.710943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.710924    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.711076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.711058    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.733338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.733307    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvbf4\" (UniqueName: \"kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4\") pod \"maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:21.783451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.783420    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:21.892666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:21.892574    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:22.053976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.053781    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:22.112732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.112707    4577 scope.go:122] "RemoveContainer" containerID="c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"
Oct 30 01:15:22.254728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.254697    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"]
Oct 30 01:15:22.255225 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:22.255197    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf852d0d3_9901_48da_af1f_f292a7731a96.slice/crio-ff619860b8cc0f7790646f630c28aef515416040f2d067b7c112c78133976c02 WatchSource:0}: Error finding container ff619860b8cc0f7790646f630c28aef515416040f2d067b7c112c78133976c02: Status 404 returned error can't find the container with id ff619860b8cc0f7790646f630c28aef515416040f2d067b7c112c78133976c02
Oct 30 01:15:22.514963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.507373    4577 generic.go:356] "Generic (PLEG): container finished" podID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerID="aed52d0e64a796ee12fcb42250831b7e79e5c6cf5aadd2bf8250c2ccea61b384" exitCode=60
Oct 30 01:15:22.514963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.507500    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerDied","Data":"aed52d0e64a796ee12fcb42250831b7e79e5c6cf5aadd2bf8250c2ccea61b384"}
Oct 30 01:15:22.514963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.507575    4577 scope.go:122] "RemoveContainer" containerID="f0123cb48dc102455847692df2ea24250b2fa86fca467d2760f941679eb22c95"
Oct 30 01:15:22.514963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.508089    4577 scope.go:122] "RemoveContainer" containerID="aed52d0e64a796ee12fcb42250831b7e79e5c6cf5aadd2bf8250c2ccea61b384"
Oct 30 01:15:22.514963 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:22.508549    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29888715-s2gfq_odh-ai-gateway-infra(e928fbec-b0ae-4f9d-ba43-d8d90a6682ba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba"
Oct 30 01:15:22.519754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.519713    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg" event={"ID":"f852d0d3-9901-48da-af1f-f292a7731a96","Type":"ContainerStarted","Data":"ff619860b8cc0f7790646f630c28aef515416040f2d067b7c112c78133976c02"}
Oct 30 01:15:22.543469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.543441    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"]
Oct 30 01:15:22.596152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.596117    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"]
Oct 30 01:15:22.887921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.863216    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:22.964989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.964955    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg\" (UniqueName: \"kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg\") pod \"f5e05e57-8619-4454-96b7-671b7d811e31\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") "
Oct 30 01:15:22.965490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.965458    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca\") pod \"f5e05e57-8619-4454-96b7-671b7d811e31\" (UID: \"f5e05e57-8619-4454-96b7-671b7d811e31\") "
Oct 30 01:15:22.966503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.966478    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca" pod "f5e05e57-8619-4454-96b7-671b7d811e31" (UID: "f5e05e57-8619-4454-96b7-671b7d811e31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:22.971924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:22.971417    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg" pod "f5e05e57-8619-4454-96b7-671b7d811e31" (UID: "f5e05e57-8619-4454-96b7-671b7d811e31"). InnerVolumeSpecName "kube-api-access-xsrsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:23.067958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.067924    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e05e57-8619-4454-96b7-671b7d811e31-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:23.067958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.067954    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsrsg\" (UniqueName: \"kubernetes.io/projected/f5e05e57-8619-4454-96b7-671b7d811e31-kube-api-access-xsrsg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:23.519334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.519243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:23.530779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.530745    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"
Oct 30 01:15:23.530970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.530776    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf" event={"ID":"f5e05e57-8619-4454-96b7-671b7d811e31","Type":"ContainerDied","Data":"71e9ab2307b8fab70f777e0a344e5ba3ccfe4928c422964da8cda54afa39cf8d"}
Oct 30 01:15:23.530970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.530855    4577 scope.go:122] "RemoveContainer" containerID="a8fc8befd93b7d7fdae657e942b8fdac6cc776cae7b5f0e03a6eb703c877a89d"
Oct 30 01:15:23.533240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.533218    4577 generic.go:356] "Generic (PLEG): container finished" podID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerID="b0f713da82f68efd3d0812b76934ee90dd91b32792b8d08a4ac69eece9cee176" exitCode=60
Oct 30 01:15:23.533363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.533297    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" event={"ID":"380dbe32-6f86-4cd5-8211-6b1621283bec","Type":"ContainerDied","Data":"b0f713da82f68efd3d0812b76934ee90dd91b32792b8d08a4ac69eece9cee176"}
Oct 30 01:15:23.533732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.533716    4577 scope.go:122] "RemoveContainer" containerID="b0f713da82f68efd3d0812b76934ee90dd91b32792b8d08a4ac69eece9cee176"
Oct 30 01:15:23.534084 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:23.534057    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9_odh-ai-gateway-infra(380dbe32-6f86-4cd5-8211-6b1621283bec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec"
Oct 30 01:15:23.535692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.535671    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg" event={"ID":"f852d0d3-9901-48da-af1f-f292a7731a96","Type":"ContainerStarted","Data":"ed211d098f722c2f4de17af9af10a15441c034c9c29bd375c39553a6c7620844"}
Oct 30 01:15:23.535814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.535801    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:23.543111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.543078    4577 scope.go:122] "RemoveContainer" containerID="c689549019304a17064258f917f2f0313208289f1eeb025e5345af0da2cd43ba"
Oct 30 01:15:23.709950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.700430    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"]
Oct 30 01:15:23.722903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.715060    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888715-c8cbf"]
Oct 30 01:15:23.776709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.776657    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg" podStartSLOduration=2.275536505 podStartE2EDuration="2.776636839s" totalImagesPullingTime="501.100334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:23.775336314 +0000 UTC m=+2031.804646546" watchObservedRunningTime="2026-10-30 01:15:23.776636839 +0000 UTC m=+2031.805947066"
Oct 30 01:15:23.811859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:23.811819    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:24.010204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.010169    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm\" (UniqueName: \"kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm\") pod \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") "
Oct 30 01:15:24.010419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.010221    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca\") pod \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\" (UID: \"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba\") "
Oct 30 01:15:24.010767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.010746    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca" pod "e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" (UID: "e928fbec-b0ae-4f9d-ba43-d8d90a6682ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:24.012451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.012429    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm" pod "e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" (UID: "e928fbec-b0ae-4f9d-ba43-d8d90a6682ba"). InnerVolumeSpecName "kube-api-access-n9jpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:24.112403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.112324    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9jpm\" (UniqueName: \"kubernetes.io/projected/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-kube-api-access-n9jpm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:24.112403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.112350    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:24.114282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.114258    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" path="/var/lib/kubelet/pods/f5e05e57-8619-4454-96b7-671b7d811e31/volumes"
Oct 30 01:15:24.208145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.208103    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:24.544390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.544362    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"
Oct 30 01:15:24.544390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.544373    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq" event={"ID":"e928fbec-b0ae-4f9d-ba43-d8d90a6682ba","Type":"ContainerDied","Data":"113977a2fc7057f4047c86e6977a8783ce7f9c5dd4861a5bde69b86236fb742f"}
Oct 30 01:15:24.544983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.544438    4577 scope.go:122] "RemoveContainer" containerID="aed52d0e64a796ee12fcb42250831b7e79e5c6cf5aadd2bf8250c2ccea61b384"
Oct 30 01:15:24.621930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.621898    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"]
Oct 30 01:15:24.625440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.625406    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"]
Oct 30 01:15:24.626343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626293    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626344    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626357    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626362    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626381    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626386    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626406    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:24.626429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626411    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:24.626781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626502    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:24.626781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626510    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626520    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626526    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:24.626781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.626534    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:24.629547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.629523    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.631773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.631750    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888715-s2gfq"]
Oct 30 01:15:24.665634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.660926    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-bbb402-openshift-default-dockercfg-skqvg\""
Oct 30 01:15:24.672346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.672311    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"]
Oct 30 01:15:24.690701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.690664    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:24.724788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724473    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.724788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724586    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.724788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724615    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.724788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724663    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.724788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724689    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.724788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724753    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kvnh\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.725531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724805    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.725531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724832    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.725531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.724924    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.830939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.828929    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kvnh\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.847510    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.847563    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.847669    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.847967    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.848119    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.848153    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.848208    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.848233    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.848727    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.849013    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.849241    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.851970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.851654    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.852758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.852388    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.865227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.863336    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.887875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.870435    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.891357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.888564    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kvnh\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:24.922494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.922452    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"]
Oct 30 01:15:24.956798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.956717    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:24.990352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:24.990319    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token\") pod \"e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") " pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:25.266798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.266771    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:25.271738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.271716    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"]
Oct 30 01:15:25.272621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.272595    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:25.272677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.272622    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" containerName="cleanup"
Oct 30 01:15:25.272724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.272685    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:25.272724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.272694    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:25.272884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.272867    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5e05e57-8619-4454-96b7-671b7d811e31" containerName="cleanup"
Oct 30 01:15:25.281896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.275496    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.288468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.288447    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:25.304795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.300548    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"]
Oct 30 01:15:25.331125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.331072    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-bdf6ca-serving-cert\""
Oct 30 01:15:25.359299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.358766    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98\" (UniqueName: \"kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98\") pod \"380dbe32-6f86-4cd5-8211-6b1621283bec\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") "
Oct 30 01:15:25.359472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.359454    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca\") pod \"380dbe32-6f86-4cd5-8211-6b1621283bec\" (UID: \"380dbe32-6f86-4cd5-8211-6b1621283bec\") "
Oct 30 01:15:25.361204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.361169    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.361341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.361269    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.361341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.361299    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5gkl\" (UniqueName: \"kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.361467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.361418    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.361921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.361893    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca" pod "380dbe32-6f86-4cd5-8211-6b1621283bec" (UID: "380dbe32-6f86-4cd5-8211-6b1621283bec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:15:25.383914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.383822    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:25.391539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.391508    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98" pod "380dbe32-6f86-4cd5-8211-6b1621283bec" (UID: "380dbe32-6f86-4cd5-8211-6b1621283bec"). InnerVolumeSpecName "kube-api-access-bgd98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:25.534529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.534487    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:25.536111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.536083    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.536553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.536529    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.536754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.536735    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.536988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.536926    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5gkl\" (UniqueName: \"kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.537228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.537206    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgd98\" (UniqueName: \"kubernetes.io/projected/380dbe32-6f86-4cd5-8211-6b1621283bec-kube-api-access-bgd98\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:25.537349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.537334    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/380dbe32-6f86-4cd5-8211-6b1621283bec-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:15:25.537453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.537415    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.538312 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:25.538290    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-serving-cert: secret "maas-api-e2e-shared-b-w3-bdf6ca-serving-cert" not found
Oct 30 01:15:25.538402 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:25.538360    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls podName:5aa19f12-8a54-4aad-9e86-b3344edac6db nodeName:}" failed. No retries permitted until 2026-10-30 01:15:26.03834085 +0000 UTC m=+2034.067651073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls") pod "maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" (UID: "5aa19f12-8a54-4aad-9e86-b3344edac6db") : secret "maas-api-e2e-shared-b-w3-bdf6ca-serving-cert" not found
Oct 30 01:15:25.612873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.612329    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5gkl\" (UniqueName: \"kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.638929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.638894    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:25.651639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.651128    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9" event={"ID":"380dbe32-6f86-4cd5-8211-6b1621283bec","Type":"ContainerDied","Data":"14f97b1b7641469fa053ca17e4ab079a1bc10d4be0df324b7c9a5117e1548758"}
Oct 30 01:15:25.651639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.651199    4577 scope.go:122] "RemoveContainer" containerID="b0f713da82f68efd3d0812b76934ee90dd91b32792b8d08a4ac69eece9cee176"
Oct 30 01:15:25.651639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.651425    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"
Oct 30 01:15:25.675501 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:25.675300    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b69ca6_e4e4_43ef_ab35_58f8d236a965.slice/crio-dcf00de23265949d1bb33035ae61302d9d679061ee3446f57b124514b2475350 WatchSource:0}: Error finding container dcf00de23265949d1bb33035ae61302d9d679061ee3446f57b124514b2475350: Status 404 returned error can't find the container with id dcf00de23265949d1bb33035ae61302d9d679061ee3446f57b124514b2475350
Oct 30 01:15:25.675754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.675728    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"]
Oct 30 01:15:25.678378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.678341    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:25.678474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.678425    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:25.678542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.678471    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:25.711768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.711743    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"]
Oct 30 01:15:25.715363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.715342    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888715-kxzs9"]
Oct 30 01:15:25.748811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:25.748784    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:26.047341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.047310    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:26.050043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.050013    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:26.113061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.113030    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" path="/var/lib/kubelet/pods/380dbe32-6f86-4cd5-8211-6b1621283bec/volumes"
Oct 30 01:15:26.113573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.113557    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e928fbec-b0ae-4f9d-ba43-d8d90a6682ba" path="/var/lib/kubelet/pods/e928fbec-b0ae-4f9d-ba43-d8d90a6682ba/volumes"
Oct 30 01:15:26.216175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.214260    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:26.634880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.627584    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"]
Oct 30 01:15:26.657501 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:26.657174    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa19f12_8a54_4aad_9e86_b3344edac6db.slice/crio-de61739131ce47fc3600a979da3be12a35d26c1f9b222829ba3dce6de34a4cee WatchSource:0}: Error finding container de61739131ce47fc3600a979da3be12a35d26c1f9b222829ba3dce6de34a4cee: Status 404 returned error can't find the container with id de61739131ce47fc3600a979da3be12a35d26c1f9b222829ba3dce6de34a4cee
Oct 30 01:15:26.668233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.668202    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" event={"ID":"32b69ca6-e4e4-43ef-ab35-58f8d236a965","Type":"ContainerStarted","Data":"b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"}
Oct 30 01:15:26.668354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.668250    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" event={"ID":"32b69ca6-e4e4-43ef-ab35-58f8d236a965","Type":"ContainerStarted","Data":"dcf00de23265949d1bb33035ae61302d9d679061ee3446f57b124514b2475350"}
Oct 30 01:15:26.719624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:26.719573    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podStartSLOduration=2.719558488 podStartE2EDuration="2.719558488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:26.716400782 +0000 UTC m=+2034.745711014" watchObservedRunningTime="2026-10-30 01:15:26.719558488 +0000 UTC m=+2034.748868719"
Oct 30 01:15:27.062476 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:27.062434    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:15:27.267470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.267440    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:27.273728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.273703    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:27.306302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.306275    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:27.679325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.679285    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" event={"ID":"5aa19f12-8a54-4aad-9e86-b3344edac6db","Type":"ContainerStarted","Data":"7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6"}
Oct 30 01:15:27.679983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.679517    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:27.679983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.679564    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" event={"ID":"5aa19f12-8a54-4aad-9e86-b3344edac6db","Type":"ContainerStarted","Data":"de61739131ce47fc3600a979da3be12a35d26c1f9b222829ba3dce6de34a4cee"}
Oct 30 01:15:27.679983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.679771    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:27.681472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.681452    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:15:27.725958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:27.725909    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" podStartSLOduration=2.298664973 podStartE2EDuration="2.725895086s" totalImagesPullingTime="427.230113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:27.724044851 +0000 UTC m=+2035.753355081" watchObservedRunningTime="2026-10-30 01:15:27.725895086 +0000 UTC m=+2035.755205318"
Oct 30 01:15:28.480870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:28.479053    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:15:29.158229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:29.156495    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:29.574635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:29.574074    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:15:30.921504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:30.921463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:32.714489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:32.714403    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:33.696387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:33.696358    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:15:34.507804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:34.507766    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:36.499403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:36.499366    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:36.665874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:36.639410    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:36.935242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:36.935137    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:37.095139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:37.095100    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:38.544685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:38.544653    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:38.866788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:38.866707    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:38.986647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:38.986609    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:39.908611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:39.908578    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:40.427049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:40.427018    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:41.965716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:41.965684    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:42.316825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:42.316792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:44.106334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:44.104774    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.922055    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"]
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923173    4577 cpu_manager.go:414] "containerMap: removing container" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923195    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923229    4577 cpu_manager.go:414] "containerMap: removing container" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923238    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923267    4577 cpu_manager.go:414] "containerMap: removing container" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923276    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923414    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923428    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.924872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.923443    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="380dbe32-6f86-4cd5-8211-6b1621283bec" containerName="cleanup"
Oct 30 01:15:45.931866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.929875    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936212    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936265    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936424    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936517    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbz8k\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936548    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936679    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936789    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936828    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.936944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.936927    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:45.938218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:45.938198    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"]
Oct 30 01:15:46.019544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.019382    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:46.038733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.038689    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.038953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.038754    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.038953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.038878    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.039112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.038965    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbz8k\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.039112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.039003    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.039112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.039074    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.039310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.039178    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.039310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.039213    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.039310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.039258    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.040215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.040188    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.040427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.040267    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.040587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.040544    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.040685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.040481    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.040775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.040695    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.238945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.238750    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbz8k\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.240988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.240013    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.241449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.241397    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.244790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.244763    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo\") pod \"e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") " pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.329856 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.329804    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:46.716675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.716650    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"]
Oct 30 01:15:46.717738 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:15:46.717711    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76329b3f_3902_43cc_83a6_a705ed42974c.slice/crio-010b570a876bbc32666e18074300385201631206b2b70d8ffa2ab004357afbca WatchSource:0}: Error finding container 010b570a876bbc32666e18074300385201631206b2b70d8ffa2ab004357afbca: Status 404 returned error can't find the container with id 010b570a876bbc32666e18074300385201631206b2b70d8ffa2ab004357afbca
Oct 30 01:15:46.720548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.720512    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:46.720652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.720587    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:46.720652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:46.720616    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:47.023181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.023133    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" event={"ID":"76329b3f-3902-43cc-83a6-a705ed42974c","Type":"ContainerStarted","Data":"3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1"}
Oct 30 01:15:47.023708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.023202    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" event={"ID":"76329b3f-3902-43cc-83a6-a705ed42974c","Type":"ContainerStarted","Data":"010b570a876bbc32666e18074300385201631206b2b70d8ffa2ab004357afbca"}
Oct 30 01:15:47.101929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.092655    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podStartSLOduration=2.092634401 podStartE2EDuration="2.092634401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:47.091591789 +0000 UTC m=+2055.120902058" watchObservedRunningTime="2026-10-30 01:15:47.092634401 +0000 UTC m=+2055.121944632"
Oct 30 01:15:47.331869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.330937    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:47.332049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.332033    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:47.332110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.332070    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:47.765870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:47.749711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:48.330330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:48.330295    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:48.330753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:48.330347    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:49.330878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:49.330218    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:49.330878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:49.330280    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:49.677347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:49.677245    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:50.330516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:50.330485    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:50.330734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:50.330539    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:51.330255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:51.330220    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:51.330648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:51.330276    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:51.559505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:51.559427    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:51.699949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:51.699793    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:52.331192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:52.331062    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:52.331192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:52.331127    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:53.331692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:53.330975    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:53.331692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:53.331056    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:53.657740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:53.657642    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:54.334734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:54.333910    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused" start-of-body=
Oct 30 01:15:54.334734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:54.333970    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.213:15021/healthz/ready\": dial tcp 10.134.0.213:15021: connect: connection refused"
Oct 30 01:15:55.335613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:55.335579    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:55.336200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:55.336036    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:55.336704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:55.336683    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:15:55.463443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:55.463327    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:56.168758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.168689    4577 generic.go:356] "Generic (PLEG): container finished" podID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerID="52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6" exitCode=0
Oct 30 01:15:56.168969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.168755    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" event={"ID":"92ddec96-952c-4a03-ae26-09aa2368fdfb","Type":"ContainerDied","Data":"52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"}
Oct 30 01:15:56.169434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.169415    4577 scope.go:122] "RemoveContainer" containerID="52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"
Oct 30 01:15:56.170951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.170932    4577 generic.go:356] "Generic (PLEG): container finished" podID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerID="96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816" exitCode=0
Oct 30 01:15:56.171053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.170981    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" event={"ID":"fe335ad9-3734-4a58-a537-f9976621b4c6","Type":"ContainerDied","Data":"96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"}
Oct 30 01:15:56.171555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.171493    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:56.171648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.171592    4577 scope.go:122] "RemoveContainer" containerID="96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"
Oct 30 01:15:56.171648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.171616    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:56.171757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.171664    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:56.173415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.173388    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:56.173491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.173459    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:56.173535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.173504    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:15:56.180703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:56.180679    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:57.018295 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:15:57.018264    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:15:57.185870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.184539    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:57.196875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.194986    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" event={"ID":"92ddec96-952c-4a03-ae26-09aa2368fdfb","Type":"ContainerStarted","Data":"4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3"}
Oct 30 01:15:57.201867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.199887    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" event={"ID":"fe335ad9-3734-4a58-a537-f9976621b4c6","Type":"ContainerStarted","Data":"266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929"}
Oct 30 01:15:57.631260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.631224    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:15:57.632780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.632752    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused" start-of-body=
Oct 30 01:15:57.632871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.632805    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused"
Oct 30 01:15:57.688868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.688140    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:57.688868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.688524    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:15:57.693874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.692936    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused" start-of-body=
Oct 30 01:15:57.693874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:57.693004    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused"
Oct 30 01:15:58.631893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:58.631831    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused" start-of-body=
Oct 30 01:15:58.632307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:58.631918    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused"
Oct 30 01:15:58.692870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:58.692305    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused" start-of-body=
Oct 30 01:15:58.692870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:58.692378    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused"
Oct 30 01:15:58.950499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:58.950370    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:15:59.306187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:59.305006    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:15:59.631701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:59.631606    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused" start-of-body=
Oct 30 01:15:59.631701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:59.631671    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused"
Oct 30 01:15:59.689870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:59.689241    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused" start-of-body=
Oct 30 01:15:59.689870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:15:59.689304    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused"
Oct 30 01:16:00.368993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:00.368955    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:16:00.632200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:00.632119    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused" start-of-body=
Oct 30 01:16:00.632200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:00.632181    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused"
Oct 30 01:16:00.690560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:00.690520    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused" start-of-body=
Oct 30 01:16:00.691234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:00.691202    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused"
Oct 30 01:16:00.712452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:00.712418    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:01.631835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:01.631799    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused" start-of-body=
Oct 30 01:16:01.632064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:01.631879    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused"
Oct 30 01:16:01.691871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:01.690377    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused" start-of-body=
Oct 30 01:16:01.691871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:01.690456    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused"
Oct 30 01:16:02.489244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.485754    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"]
Oct 30 01:16:02.530085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.530053    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"]
Oct 30 01:16:02.531327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.530892    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.552870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.550000    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-bbb402-serving-cert\""
Oct 30 01:16:02.624014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.620377    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:02.631476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.631445    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused" start-of-body=
Oct 30 01:16:02.631636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.631508    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.201:15021/healthz/ready\": dial tcp 10.134.0.201:15021: connect: connection refused"
Oct 30 01:16:02.632814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.632787    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.633067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.633044    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.633226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.633203    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.633395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.633372    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f524n\" (UniqueName: \"kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.695447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.695408    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused" start-of-body=
Oct 30 01:16:02.696080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.696046    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.202:15021/healthz/ready\": dial tcp 10.134.0.202:15021: connect: connection refused"
Oct 30 01:16:02.770932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.745995    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.784231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.784113    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f524n\" (UniqueName: \"kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.784754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.784723    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.785105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.785083    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.804176 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:16:02.790684    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-serving-cert: secret "maas-api-e2e-explicit-bbb402-serving-cert" not found
Oct 30 01:16:02.804176 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:16:02.790793    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls podName:2c18f8d1-8606-468f-b533-ad1552efa6f2 nodeName:}" failed. No retries permitted until 2026-10-30 01:16:03.290759968 +0000 UTC m=+2071.320070193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls") pod "maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" (UID: "2c18f8d1-8606-468f-b533-ad1552efa6f2") : secret "maas-api-e2e-explicit-bbb402-serving-cert" not found
Oct 30 01:16:02.804176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.792694    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.808677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.808632    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:02.815008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.814978    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f524n\" (UniqueName: \"kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.817451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.817411    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:02.962869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:02.961820    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:03.309969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.308679    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:03.316218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.314987    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\") pod \"maas-api-e2e-explicit-bbb402-58949778f7-4rzdn\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:03.530069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.530035    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:03.638013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.637812    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:16:03.639249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.639224    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:16:03.833166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.823187    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:16:03.853098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.839787    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:16:03.873785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.872333    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"]
Oct 30 01:16:03.877920 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:16:03.877402    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c18f8d1_8606_468f_b533_ad1552efa6f2.slice/crio-4cc863d55087cbe304468fdbce421d56240a224692b3dfeddaa63244978ef3ed WatchSource:0}: Error finding container 4cc863d55087cbe304468fdbce421d56240a224692b3dfeddaa63244978ef3ed: Status 404 returned error can't find the container with id 4cc863d55087cbe304468fdbce421d56240a224692b3dfeddaa63244978ef3ed
Oct 30 01:16:03.880937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:03.879821    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:16:04.331604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:04.331567    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" event={"ID":"2c18f8d1-8606-468f-b533-ad1552efa6f2","Type":"ContainerStarted","Data":"4cc863d55087cbe304468fdbce421d56240a224692b3dfeddaa63244978ef3ed"}
Oct 30 01:16:04.412620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:04.412583    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:05.344540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:05.344498    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" event={"ID":"2c18f8d1-8606-468f-b533-ad1552efa6f2","Type":"ContainerStarted","Data":"64c05af6b3bceff83033812d2e9505b63f06ed134e781c570d70959f0a4b3916"}
Oct 30 01:16:05.345013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:05.344573    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:05.395713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:05.395652    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" podStartSLOduration=2.994722001 podStartE2EDuration="3.395637863s" totalImagesPullingTime="400.915862ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:16:05.394104616 +0000 UTC m=+2073.423414846" watchObservedRunningTime="2026-10-30 01:16:05.395637863 +0000 UTC m=+2073.424948094"
Oct 30 01:16:06.252633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:06.252583    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:06.878893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:06.878099    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:08.119956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:08.119925    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:08.868444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:08.868406    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:09.896110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:09.896078    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:11.395495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:11.395461    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:16:11.738805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:11.738770    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:13.401035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:13.400991    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:16:13.401572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:13.401068    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:16:13.548125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:13.548088    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:15.295201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.295158    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:15.473400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.473361    4577 generic.go:356] "Generic (PLEG): container finished" podID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerID="bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3" exitCode=0
Oct 30 01:16:15.473582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.473441    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" event={"ID":"9998a2ee-6eba-4438-8713-af86376d2dc2","Type":"ContainerDied","Data":"bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"}
Oct 30 01:16:15.474087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.474067    4577 scope.go:122] "RemoveContainer" containerID="bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"
Oct 30 01:16:15.476110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.476080    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:15.476225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.476145    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:15.476225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:15.476176    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:16.486907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:16.486870    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" event={"ID":"9998a2ee-6eba-4438-8713-af86376d2dc2","Type":"ContainerStarted","Data":"91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22"}
Oct 30 01:16:16.554308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:16.554270    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:16:16.554713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:16.554616    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused" start-of-body=
Oct 30 01:16:16.554926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:16.554901    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused"
Oct 30 01:16:17.185815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:17.185752    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:17.320274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:17.320237    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:17.542872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:17.542817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:17.557294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:17.557257    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused" start-of-body=
Oct 30 01:16:17.557600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:17.557569    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused"
Oct 30 01:16:17.701804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:17.701704    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:18.556870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:18.555379    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused" start-of-body=
Oct 30 01:16:18.556870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:18.555443    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused"
Oct 30 01:16:19.006058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:19.006024    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:19.554271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:19.554231    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused" start-of-body=
Oct 30 01:16:19.554577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:19.554546    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused"
Oct 30 01:16:20.576431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:20.576392    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused" start-of-body=
Oct 30 01:16:20.577043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:20.576456    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused"
Oct 30 01:16:20.824870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:20.824011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:21.561872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:21.558249    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused" start-of-body=
Oct 30 01:16:21.561872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:21.558313    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.207:15021/healthz/ready\": dial tcp 10.134.0.207:15021: connect: connection refused"
Oct 30 01:16:22.265763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:22.265720    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:22.561252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:22.561168    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:16:22.563057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:22.563028    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:16:22.563785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:22.563764    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:16:22.605293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:22.605252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:24.428799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:24.428764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:26.343825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:26.343789    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:27.014675 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:16:27.014641    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:16:28.118512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:28.118479    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:28.506082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:28.506050    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:30.002296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:30.002248    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:31.711485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:31.711447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:32.873814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:32.873730    4577 generic.go:356] "Generic (PLEG): container finished" podID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerID="b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb" exitCode=0
Oct 30 01:16:32.873814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:32.873772    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" event={"ID":"32b69ca6-e4e4-43ef-ab35-58f8d236a965","Type":"ContainerDied","Data":"b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"}
Oct 30 01:16:32.874402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:32.874228    4577 scope.go:122] "RemoveContainer" containerID="b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"
Oct 30 01:16:32.875919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:32.875889    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:32.876002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:32.875968    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:32.876060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:32.876001    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:33.612049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:33.612019    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:33.895179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:33.895082    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" event={"ID":"32b69ca6-e4e4-43ef-ab35-58f8d236a965","Type":"ContainerStarted","Data":"53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd"}
Oct 30 01:16:34.273869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:34.272344    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:16:34.273869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:34.272477    4577 patch_prober.go:29] interesting pod/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused" start-of-body=
Oct 30 01:16:34.273869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:34.272522    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused"
Oct 30 01:16:35.271443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:35.271393    4577 patch_prober.go:29] interesting pod/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused" start-of-body=
Oct 30 01:16:35.272131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:35.272080    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused"
Oct 30 01:16:35.309878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:35.309829    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:36.272870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:36.272313    4577 patch_prober.go:29] interesting pod/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused" start-of-body=
Oct 30 01:16:36.272870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:36.272382    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused"
Oct 30 01:16:37.028629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:37.028588    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:37.280632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:37.280152    4577 patch_prober.go:29] interesting pod/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused" start-of-body=
Oct 30 01:16:37.280632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:37.280208    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused"
Oct 30 01:16:38.270773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:38.270301    4577 patch_prober.go:29] interesting pod/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused" start-of-body=
Oct 30 01:16:38.270773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:38.270381    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused"
Oct 30 01:16:38.718908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:38.718802    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:39.267785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:39.267743    4577 patch_prober.go:29] interesting pod/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused" start-of-body=
Oct 30 01:16:39.268021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:39.267816    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.211:15021/healthz/ready\": dial tcp 10.134.0.211:15021: connect: connection refused"
Oct 30 01:16:40.273524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:40.273491    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:16:40.273961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:40.273880    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:16:40.274696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:40.274675    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:16:40.609938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:40.605081    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:42.263047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:42.263012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:42.993788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:42.993734    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:43.998443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:43.998405    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:44.996082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:44.996039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:45.708291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:45.708252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:47.499189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:47.499148    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:49.217531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:49.217490    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:51.136873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:51.136807    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:51.692015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:51.691978    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:52.524384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:52.524345    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.196:15021/healthz/ready\": read tcp 10.134.0.2:53870->10.134.0.196:15021: read: connection reset by peer" start-of-body=
Oct 30 01:16:52.525620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:52.524430    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.196:15021/healthz/ready\": read tcp 10.134.0.2:53870->10.134.0.196:15021: read: connection reset by peer"
Oct 30 01:16:53.016960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.016922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:53.137624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.137587    4577 generic.go:356] "Generic (PLEG): container finished" podID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerID="2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9" exitCode=0
Oct 30 01:16:53.137897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.137754    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" event={"ID":"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d","Type":"ContainerDied","Data":"2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"}
Oct 30 01:16:53.138220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.138197    4577 scope.go:122] "RemoveContainer" containerID="2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"
Oct 30 01:16:53.140821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.140752    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:53.141000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.140859    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:53.141000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:53.140911    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:16:54.173429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:54.173389    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" event={"ID":"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d","Type":"ContainerStarted","Data":"585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9"}
Oct 30 01:16:54.743413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:54.743385    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:16:54.743598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:54.743537    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused" start-of-body=
Oct 30 01:16:54.743642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:54.743593    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused"
Oct 30 01:16:54.993548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:54.993466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:55.743930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:55.743898    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused" start-of-body=
Oct 30 01:16:55.744428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:55.743958    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused"
Oct 30 01:16:56.737209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:56.737178    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:56.743584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:56.743556    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused" start-of-body=
Oct 30 01:16:56.743729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:56.743620    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused"
Oct 30 01:16:57.017506 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:16:57.017422    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:16:57.576453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:57.576409    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:57.743641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:57.743602    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused" start-of-body=
Oct 30 01:16:57.743867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:57.743660    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused"
Oct 30 01:16:58.633372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:58.633335    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:16:58.745733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:58.745236    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused" start-of-body=
Oct 30 01:16:58.745733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:58.745307    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.196:15021/healthz/ready\": dial tcp 10.134.0.196:15021: connect: connection refused"
Oct 30 01:16:59.774433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:59.774209    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:16:59.776887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:59.775811    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:16:59.777954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:16:59.777927    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:17:00.354771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:00.354738    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:00.984482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:00.984452    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:01.101155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:01.101125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:02.042211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:02.042168    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:03.825177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:03.825148    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:05.718220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:05.718182    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:07.492972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:07.492941    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:08.410096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:08.410065    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:09.323609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:09.323577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:11.166867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:11.166347    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:11.968097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:11.968061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:13.025243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:13.025210    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:14.775822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:14.775787    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:16.596349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:16.596306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:17.290555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.290517    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"]
Oct 30 01:17:17.296591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.296567    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.312443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.312416    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"]
Oct 30 01:17:17.321820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.321787    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"openshift-service-ca.crt\""
Oct 30 01:17:17.321986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.321949    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"default-dockercfg-ndxl7\""
Oct 30 01:17:17.327222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.327200    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"kube-root-ca.crt\""
Oct 30 01:17:17.327547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.327527    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"auth-test-model-9da49f-kserve-self-signed-certs\""
Oct 30 01:17:17.333758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.333734    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:17.367365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.367332    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.367609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.367427    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.367609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.367518    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.367609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.367590    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.367801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.367713    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djq6f\" (UniqueName: \"kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.367801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.367777    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.469812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.469773    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djq6f\" (UniqueName: \"kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.469898    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470103    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470183    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470220    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470278    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470374    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470424    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.470670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.470646    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.472735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.472711    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.473044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.473024    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.495015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.494990    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djq6f\" (UniqueName: \"kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f\") pod \"auth-test-model-9da49f-kserve-845b7c99bf-z9rcg\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:17.614919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:17.614225    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:18.011319 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:18.011291    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"]
Oct 30 01:17:18.012193 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:17:18.012167    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46854d92_0409_4d5d_aaf3_27316bc1838f.slice/crio-d279caa18a77c1da1bd1d961d19ebc184bbf53cef05d236f83d976c9ffba4af2 WatchSource:0}: Error finding container d279caa18a77c1da1bd1d961d19ebc184bbf53cef05d236f83d976c9ffba4af2: Status 404 returned error can't find the container with id d279caa18a77c1da1bd1d961d19ebc184bbf53cef05d236f83d976c9ffba4af2
Oct 30 01:17:18.363524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:18.363414    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg" event={"ID":"46854d92-0409-4d5d-aaf3-27316bc1838f","Type":"ContainerStarted","Data":"9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5"}
Oct 30 01:17:18.363524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:18.363470    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg" event={"ID":"46854d92-0409-4d5d-aaf3-27316bc1838f","Type":"ContainerStarted","Data":"d279caa18a77c1da1bd1d961d19ebc184bbf53cef05d236f83d976c9ffba4af2"}
Oct 30 01:17:18.409876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:18.409804    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg" podStartSLOduration=1.409791611 podStartE2EDuration="1.409791611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:17:18.40791363 +0000 UTC m=+2146.437223864" watchObservedRunningTime="2026-10-30 01:17:18.409791611 +0000 UTC m=+2146.439101842"
Oct 30 01:17:18.419665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:18.419639    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:18.615467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:18.615381    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:20.227061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:20.227029    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:22.017111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:22.017073    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:22.377203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:22.377102    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:23.131731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:23.131696    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:24.014086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:24.014051    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:25.922565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:25.922529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:27.016324 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:17:27.016282    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:17:27.244992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:27.244962    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:27.621302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:27.621267    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:27.651515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:27.651485    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:27.835694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:27.835653    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:28.498263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:28.498235    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:17:29.600025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:29.599992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:31.399511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:31.399476    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:33.304770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:33.304737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:33.961984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:33.961954    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:34.092458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:34.088927    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:34.999563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:34.999526    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:36.664543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:36.664507    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:38.394634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:38.394596    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:40.182571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:40.182525    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:40.777176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:40.776096    4577 scope.go:122] "RemoveContainer" containerID="bb1f8866d84dc06750b7d33c50c74e88294103c4f16fe0ed5548b6af8a32394d"
Oct 30 01:17:41.424292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:41.424252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:42.009917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:42.009871    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:43.891268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:43.891231    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:45.632858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:45.632810    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:47.396261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:47.396231    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:49.212625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:49.212585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:50.611905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:50.611868    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"]
Oct 30 01:17:50.612359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:50.612113    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" podUID="2c18f8d1-8606-468f-b533-ad1552efa6f2" containerName="maas-api" containerID="cri-o://64c05af6b3bceff83033812d2e9505b63f06ed134e781c570d70959f0a4b3916" gracePeriod=30
Oct 30 01:17:50.673139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:50.673095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:50.768328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:50.768295    4577 generic.go:356] "Generic (PLEG): container finished" podID="2c18f8d1-8606-468f-b533-ad1552efa6f2" containerID="64c05af6b3bceff83033812d2e9505b63f06ed134e781c570d70959f0a4b3916" exitCode=0
Oct 30 01:17:50.768515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:50.768363    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" event={"ID":"2c18f8d1-8606-468f-b533-ad1552efa6f2","Type":"ContainerDied","Data":"64c05af6b3bceff83033812d2e9505b63f06ed134e781c570d70959f0a4b3916"}
Oct 30 01:17:51.211948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.207582    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:17:51.275446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.275363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:51.334912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.327533    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\") pod \"2c18f8d1-8606-468f-b533-ad1552efa6f2\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") "
Oct 30 01:17:51.334912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.327591    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n\" (UniqueName: \"kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n\") pod \"2c18f8d1-8606-468f-b533-ad1552efa6f2\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") "
Oct 30 01:17:51.334912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.327687    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca\") pod \"2c18f8d1-8606-468f-b533-ad1552efa6f2\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") "
Oct 30 01:17:51.334912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.328285    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs\") pod \"2c18f8d1-8606-468f-b533-ad1552efa6f2\" (UID: \"2c18f8d1-8606-468f-b533-ad1552efa6f2\") "
Oct 30 01:17:51.334912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.332319    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca" pod "2c18f8d1-8606-468f-b533-ad1552efa6f2" (UID: "2c18f8d1-8606-468f-b533-ad1552efa6f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:17:51.361183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.354292    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n" pod "2c18f8d1-8606-468f-b533-ad1552efa6f2" (UID: "2c18f8d1-8606-468f-b533-ad1552efa6f2"). InnerVolumeSpecName "kube-api-access-f524n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:17:51.361183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.355700    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls" pod "2c18f8d1-8606-468f-b533-ad1552efa6f2" (UID: "2c18f8d1-8606-468f-b533-ad1552efa6f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:17:51.361183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.355928    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs" pod "2c18f8d1-8606-468f-b533-ad1552efa6f2" (UID: "2c18f8d1-8606-468f-b533-ad1552efa6f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:17:51.389333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.389290    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:51.430777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.430743    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:17:51.430777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.430771    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c18f8d1-8606-468f-b533-ad1552efa6f2-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:17:51.430777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.430780    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f524n\" (UniqueName: \"kubernetes.io/projected/2c18f8d1-8606-468f-b533-ad1552efa6f2-kube-api-access-f524n\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:17:51.431038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.430789    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c18f8d1-8606-468f-b533-ad1552efa6f2-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:17:51.503927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.503891    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:51.780448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.780412    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"
Oct 30 01:17:51.780448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.780437    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn" event={"ID":"2c18f8d1-8606-468f-b533-ad1552efa6f2","Type":"ContainerDied","Data":"4cc863d55087cbe304468fdbce421d56240a224692b3dfeddaa63244978ef3ed"}
Oct 30 01:17:51.781049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.780511    4577 scope.go:122] "RemoveContainer" containerID="64c05af6b3bceff83033812d2e9505b63f06ed134e781c570d70959f0a4b3916"
Oct 30 01:17:51.839753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.839724    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"]
Oct 30 01:17:51.854953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.854926    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bbb402-58949778f7-4rzdn"]
Oct 30 01:17:51.916683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:51.916654    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:52.113371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:52.113275    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c18f8d1-8606-468f-b533-ad1552efa6f2" path="/var/lib/kubelet/pods/2c18f8d1-8606-468f-b533-ad1552efa6f2/volumes"
Oct 30 01:17:53.239241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:53.239200    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:55.051242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:55.051190    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:56.844151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:56.844121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:57.014176 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:17:57.014138    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:17:58.364510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:58.364439    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:58.483988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:58.483960    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:58.577417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:58.577381    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"]
Oct 30 01:17:58.577729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:58.577698    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy" containerID="cri-o://53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd" gracePeriod=30
Oct 30 01:17:58.633280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:58.633205    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:58.761473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:58.761366    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:59.212193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.212160    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:59.723748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.723715    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"]
Oct 30 01:17:59.725045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.725020    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2c18f8d1-8606-468f-b533-ad1552efa6f2" containerName="maas-api"
Oct 30 01:17:59.725045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.725044    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c18f8d1-8606-468f-b533-ad1552efa6f2" containerName="maas-api"
Oct 30 01:17:59.725259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.725245    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c18f8d1-8606-468f-b533-ad1552efa6f2" containerName="maas-api"
Oct 30 01:17:59.728944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.728921    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.751322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.751288    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"]
Oct 30 01:17:59.758076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.758048    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-6d45f8-openshift-default-dockercfg-k42gj\""
Oct 30 01:17:59.769461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.769416    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:59.785285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785246    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785326    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwgsc\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785424    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785464    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785503    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785602    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785709    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785757    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.785854 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.785827    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.882097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.882064    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:17:59.887886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.887831    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.887984    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888030    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888093    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888164    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888189    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888235    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwgsc\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888325    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888362    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888399    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888425    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.888967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.888945    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.889180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.889017    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.889297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.889163    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.890655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.890627    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.890839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.890820    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.930867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.930762    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwgsc\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:17:59.952543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:17:59.952510    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token\") pod \"e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") " pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:00.060905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.059146    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:00.544315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.544282    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:00.548472 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:18:00.548449    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fb2128_7e2f_410f_8a77_b4782ef926b7.slice/crio-69c8cd9bbb54bf9559f719645f3c7252527f4972280c0b4445dc6f7f9e744e36 WatchSource:0}: Error finding container 69c8cd9bbb54bf9559f719645f3c7252527f4972280c0b4445dc6f7f9e744e36: Status 404 returned error can't find the container with id 69c8cd9bbb54bf9559f719645f3c7252527f4972280c0b4445dc6f7f9e744e36
Oct 30 01:18:00.548585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.548566    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"]
Oct 30 01:18:00.551004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.550972    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:18:00.551104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.551064    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:18:00.551171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.551110    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:18:00.872718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.872611    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" event={"ID":"39fb2128-7e2f-410f-8a77-b4782ef926b7","Type":"ContainerStarted","Data":"a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a"}
Oct 30 01:18:00.872718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.872676    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" event={"ID":"39fb2128-7e2f-410f-8a77-b4782ef926b7","Type":"ContainerStarted","Data":"69c8cd9bbb54bf9559f719645f3c7252527f4972280c0b4445dc6f7f9e744e36"}
Oct 30 01:18:00.922572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:00.922516    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" podStartSLOduration=1.92250103 podStartE2EDuration="1.92250103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:18:00.921535123 +0000 UTC m=+2188.950845569" watchObservedRunningTime="2026-10-30 01:18:00.92250103 +0000 UTC m=+2188.951811260"
Oct 30 01:18:01.074872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:01.067269    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:01.087876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:01.086021    4577 patch_prober.go:29] interesting pod/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.216:15021/healthz/ready\": dial tcp 10.134.0.216:15021: connect: connection refused" start-of-body=
Oct 30 01:18:01.087876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:01.086106    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.216:15021/healthz/ready\": dial tcp 10.134.0.216:15021: connect: connection refused"
Oct 30 01:18:01.295358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:01.295319    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:01.421509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:01.421460    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:01.587654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:01.587579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:02.065325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:02.065286    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:02.464792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:02.464692    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:02.883195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:02.883036    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:02.888832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:02.888805    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:02.890048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:02.890028    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:03.875373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.875346    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:18:03.897757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.897725    4577 generic.go:356] "Generic (PLEG): container finished" podID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerID="53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd" exitCode=0
Oct 30 01:18:03.897966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.897778    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"
Oct 30 01:18:03.897966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.897811    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" event={"ID":"32b69ca6-e4e4-43ef-ab35-58f8d236a965","Type":"ContainerDied","Data":"53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd"}
Oct 30 01:18:03.897966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.897906    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh" event={"ID":"32b69ca6-e4e4-43ef-ab35-58f8d236a965","Type":"ContainerDied","Data":"dcf00de23265949d1bb33035ae61302d9d679061ee3446f57b124514b2475350"}
Oct 30 01:18:03.897966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.897936    4577 scope.go:122] "RemoveContainer" containerID="53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd"
Oct 30 01:18:03.908947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.908923    4577 scope.go:122] "RemoveContainer" containerID="b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"
Oct 30 01:18:03.909128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909103    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909138    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909259    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909305    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909369    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909398    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909470    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909538    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909602    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo\") pod \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\" (UID: \"32b69ca6-e4e4-43ef-ab35-58f8d236a965\") "
Oct 30 01:18:03.909683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909655    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:03.909906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.909815    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910090    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910119    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910289    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910692    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910714    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910728    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910746    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:03.910974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.910760    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:03.912164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.912088    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "kube-api-access-6kvnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:18:03.912272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.912214    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:03.912580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.912555    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:18:03.912829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:03.912808    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo" pod "32b69ca6-e4e4-43ef-ab35-58f8d236a965" (UID: "32b69ca6-e4e4-43ef-ab35-58f8d236a965"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:18:04.013937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.013028    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:04.013937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.013077    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kvnh\" (UniqueName: \"kubernetes.io/projected/32b69ca6-e4e4-43ef-ab35-58f8d236a965-kube-api-access-6kvnh\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:04.013937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.013091    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:04.013937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.013111    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b69ca6-e4e4-43ef-ab35-58f8d236a965-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:04.029153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.029029    4577 scope.go:122] "RemoveContainer" containerID="53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd"
Oct 30 01:18:04.029478 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:18:04.029451    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd\": container with ID starting with 53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd not found: ID does not exist" containerID="53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd"
Oct 30 01:18:04.029586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.029494    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd"} err="failed to get container status \"53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd\": rpc error: code = NotFound desc = could not find container \"53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd\": container with ID starting with 53c5f2466eeb10842613bce020f954d8b2ef1e404b0d3477bd4fac202ff7c3fd not found: ID does not exist"
Oct 30 01:18:04.029586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.029518    4577 scope.go:122] "RemoveContainer" containerID="b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"
Oct 30 01:18:04.030261 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:18:04.030234    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb\": container with ID starting with b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb not found: ID does not exist" containerID="b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"
Oct 30 01:18:04.030373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.030267    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb"} err="failed to get container status \"b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb\": rpc error: code = NotFound desc = could not find container \"b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb\": container with ID starting with b5f520203c34d5cd35800cb0c9c34fef47b114c6d2b1b611d697c76dcaa364cb not found: ID does not exist"
Oct 30 01:18:04.245579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.245546    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"]
Oct 30 01:18:04.250063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.250032    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-bbb402-openshift-default-66b6bcd4b5-phzvh"]
Oct 30 01:18:04.307113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:04.307077    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:05.989671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:05.989634    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:06.131168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:06.131136    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" path="/var/lib/kubelet/pods/32b69ca6-e4e4-43ef-ab35-58f8d236a965/volumes"
Oct 30 01:18:07.719820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:07.719784    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:09.520816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:09.520782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:11.220210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:11.220179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:13.018369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:13.018268    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:14.706603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:14.706570    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:16.433561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:16.433523    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:18.299394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:18.299355    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:20.058838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:20.058799    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:21.843139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:21.843103    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:22.494548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:22.494502    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:23.604913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:23.604875    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:24.550324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:24.550287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:25.421377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:25.421345    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:27.014973 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:18:27.014938    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:18:27.198672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:27.198641    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:28.946669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:28.946627    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:30.734229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:30.734196    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:31.125125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:31.125047    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:31.345881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:31.341980    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"]
Oct 30 01:18:31.345881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:31.342285    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerName="istio-proxy" containerID="cri-o://a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a" gracePeriod=30
Oct 30 01:18:31.389274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:31.389166    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:31.670160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:31.670041    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:31.781148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:31.781099    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:32.669215 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:32.669125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:32.892100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:32.892061    4577 patch_prober.go:29] interesting pod/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:18:32.892506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:32.892143    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:18:34.435678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:34.435645    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:36.160382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.160341    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:36.559525 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:18:36.559485    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fb2128_7e2f_410f_8a77_b4782ef926b7.slice/crio-conmon-a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fb2128_7e2f_410f_8a77_b4782ef926b7.slice/crio-69c8cd9bbb54bf9559f719645f3c7252527f4972280c0b4445dc6f7f9e744e36\": RecentStats: unable to find data in memory cache]"
Oct 30 01:18:36.743443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.743417    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:36.775979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.775948    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776040    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776143    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776185    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776228    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776293    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:36.776365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776298    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776475    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776541    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.776638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776626    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert\") pod \"39fb2128-7e2f-410f-8a77-b4782ef926b7\" (UID: \"39fb2128-7e2f-410f-8a77-b4782ef926b7\") "
Oct 30 01:18:36.777253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.776950    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:36.777253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777134    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:36.777253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777206    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:36.777505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777259    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:18:36.777610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777588    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.777658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777619    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.777658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777635    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39fb2128-7e2f-410f-8a77-b4782ef926b7-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.777658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777649    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.777773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.777661    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.778664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.778628    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:18:36.778887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.778830    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:18:36.778975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.778945    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:18:36.779187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.779171    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc" pod "39fb2128-7e2f-410f-8a77-b4782ef926b7" (UID: "39fb2128-7e2f-410f-8a77-b4782ef926b7"). InnerVolumeSpecName "kube-api-access-mwgsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:18:36.793768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.793739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:36.878944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.878915    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.878944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.878941    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwgsc\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-kube-api-access-mwgsc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.879160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.878954    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:36.879160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:36.878969    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39fb2128-7e2f-410f-8a77-b4782ef926b7-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:18:37.190151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.190066    4577 generic.go:356] "Generic (PLEG): container finished" podID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerID="a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a" exitCode=0
Oct 30 01:18:37.190618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.190166    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"
Oct 30 01:18:37.190618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.190181    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" event={"ID":"39fb2128-7e2f-410f-8a77-b4782ef926b7","Type":"ContainerDied","Data":"a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a"}
Oct 30 01:18:37.190618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.190229    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6" event={"ID":"39fb2128-7e2f-410f-8a77-b4782ef926b7","Type":"ContainerDied","Data":"69c8cd9bbb54bf9559f719645f3c7252527f4972280c0b4445dc6f7f9e744e36"}
Oct 30 01:18:37.190618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.190250    4577 scope.go:122] "RemoveContainer" containerID="a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a"
Oct 30 01:18:37.216037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.216015    4577 scope.go:122] "RemoveContainer" containerID="a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a"
Oct 30 01:18:37.216363 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:18:37.216345    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a\": container with ID starting with a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a not found: ID does not exist" containerID="a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a"
Oct 30 01:18:37.216439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.216374    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a"} err="failed to get container status \"a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a\": rpc error: code = NotFound desc = could not find container \"a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a\": container with ID starting with a6a2a421226c9ce0d608ea48f40887c20c9e387adaabb0e9bc647c8d32a36c6a not found: ID does not exist"
Oct 30 01:18:37.251333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.251301    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"]
Oct 30 01:18:37.257063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.257031    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-6d45f8-openshift-default-7b5d47d899-7p2c6"]
Oct 30 01:18:37.924350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:37.924312    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:38.114226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:38.114185    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" path="/var/lib/kubelet/pods/39fb2128-7e2f-410f-8a77-b4782ef926b7/volumes"
Oct 30 01:18:39.719450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:39.719414    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:41.426031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:41.425989    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:43.218993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:43.218962    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:45.016585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:45.016548    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:46.784658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:46.784616    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:48.762690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:48.762652    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:50.516040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:50.516005    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:52.262203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:52.262162    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:53.946326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:53.946285    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:55.479730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:55.479695    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"]
Oct 30 01:18:55.480186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:55.479989    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy" containerID="cri-o://2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0" gracePeriod=30
Oct 30 01:18:55.496446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:55.496411    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"]
Oct 30 01:18:55.496688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:55.496657    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy" containerID="cri-o://273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786" gracePeriod=30
Oct 30 01:18:55.799524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:55.799436    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:57.014142 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:18:57.014097    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:18:57.496281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:57.496247    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:18:59.315219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:18:59.315185    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:00.791801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.791772    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:19:00.842587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.842561    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:19:00.908662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.908623    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.908898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.908767    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.908898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.908833    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.908898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.908888    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.909170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.908916    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.909007    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.914077    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.914256    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.914313    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.914797    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "kube-api-access-98pqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.915015    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.915053    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.915118    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy\") pod \"333870ba-945f-4e5c-949a-ca525c615630\" (UID: \"333870ba-945f-4e5c-949a-ca525c615630\") "
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.915861    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.916741    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98pqc\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-kube-api-access-98pqc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.916762    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.916777    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.916215    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:00.916906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.916479    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:19:00.919596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.919560    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:00.920072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.920047    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:19:00.921273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:00.921246    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token" pod "333870ba-945f-4e5c-949a-ca525c615630" (UID: "333870ba-945f-4e5c-949a-ca525c615630"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.018161    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.018619    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.018718    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.019325    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.021364    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.021520    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.021748    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.021801    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.021910    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.022069    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr\") pod \"a5d9c977-a868-48ad-86df-25432459b386\" (UID: \"a5d9c977-a868-48ad-86df-25432459b386\") "
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023002    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023414    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023517    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023542    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/333870ba-945f-4e5c-949a-ca525c615630-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023556    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023568    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/333870ba-945f-4e5c-949a-ca525c615630-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023581    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/333870ba-945f-4e5c-949a-ca525c615630-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023594    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.023692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023607    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/333870ba-945f-4e5c-949a-ca525c615630-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023701    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:01.024862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.023969    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:01.026394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.026365    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:01.026932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.026907    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:19:01.027040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.027020    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:01.027950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.027919    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr" pod "a5d9c977-a868-48ad-86df-25432459b386" (UID: "a5d9c977-a868-48ad-86df-25432459b386"). InnerVolumeSpecName "kube-api-access-cx8fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:01.125814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125780    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx8fr\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-kube-api-access-cx8fr\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.125814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125814    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d9c977-a868-48ad-86df-25432459b386-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.126037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125830    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.126037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125872    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d9c977-a868-48ad-86df-25432459b386-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.126037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125886    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.126037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125900    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d9c977-a868-48ad-86df-25432459b386-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.126037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.125916    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d9c977-a868-48ad-86df-25432459b386-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:01.180257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.180221    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:01.462089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.461993    4577 generic.go:356] "Generic (PLEG): container finished" podID="a5d9c977-a868-48ad-86df-25432459b386" containerID="273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786" exitCode=0
Oct 30 01:19:01.462089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.462044    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"
Oct 30 01:19:01.462326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.462086    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" event={"ID":"a5d9c977-a868-48ad-86df-25432459b386","Type":"ContainerDied","Data":"273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786"}
Oct 30 01:19:01.462326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.462141    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257" event={"ID":"a5d9c977-a868-48ad-86df-25432459b386","Type":"ContainerDied","Data":"c024ca445d81db96837bc4a88d8204353ea14392c7fed19709a8df3e47283523"}
Oct 30 01:19:01.462326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.462175    4577 scope.go:122] "RemoveContainer" containerID="273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786"
Oct 30 01:19:01.464185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.464162    4577 generic.go:356] "Generic (PLEG): container finished" podID="333870ba-945f-4e5c-949a-ca525c615630" containerID="2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0" exitCode=0
Oct 30 01:19:01.464362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.464224    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"
Oct 30 01:19:01.464362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.464329    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" event={"ID":"333870ba-945f-4e5c-949a-ca525c615630","Type":"ContainerDied","Data":"2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0"}
Oct 30 01:19:01.464501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.464365    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht" event={"ID":"333870ba-945f-4e5c-949a-ca525c615630","Type":"ContainerDied","Data":"1863e62a560b511236e13f0ee516ade64838236ad7521a4b6b94ca9bd7e7a258"}
Oct 30 01:19:01.473541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.473519    4577 scope.go:122] "RemoveContainer" containerID="273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786"
Oct 30 01:19:01.473816 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:01.473795    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786\": container with ID starting with 273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786 not found: ID does not exist" containerID="273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786"
Oct 30 01:19:01.473949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.473828    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786"} err="failed to get container status \"273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786\": rpc error: code = NotFound desc = could not find container \"273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786\": container with ID starting with 273cef643fee961f855331b875339099dace4aeb48a88b323c6c340e573a7786 not found: ID does not exist"
Oct 30 01:19:01.473949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.473890    4577 scope.go:122] "RemoveContainer" containerID="2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0"
Oct 30 01:19:01.484086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.484065    4577 scope.go:122] "RemoveContainer" containerID="2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0"
Oct 30 01:19:01.484399 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:01.484383    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0\": container with ID starting with 2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0 not found: ID does not exist" containerID="2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0"
Oct 30 01:19:01.484458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.484406    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0"} err="failed to get container status \"2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0\": rpc error: code = NotFound desc = could not find container \"2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0\": container with ID starting with 2f2ab1115848588efa47c4c5c7707b572520272d9c93f0cc729e0a134d6770e0 not found: ID does not exist"
Oct 30 01:19:01.524576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.524545    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"]
Oct 30 01:19:01.535779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.535752    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-5r257"]
Oct 30 01:19:01.581915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.581879    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"]
Oct 30 01:19:01.587420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:01.587353    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-qclht"]
Oct 30 01:19:02.113917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:02.113886    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="333870ba-945f-4e5c-949a-ca525c615630" path="/var/lib/kubelet/pods/333870ba-945f-4e5c-949a-ca525c615630/volumes"
Oct 30 01:19:02.114434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:02.114417    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5d9c977-a868-48ad-86df-25432459b386" path="/var/lib/kubelet/pods/a5d9c977-a868-48ad-86df-25432459b386/volumes"
Oct 30 01:19:03.024321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:03.024287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:05.008978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:05.008942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:06.717047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:06.716996    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:08.496468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:08.496433    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:10.214669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:10.214636    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:12.005164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:12.005123    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:13.721279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:13.721240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:14.196818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.196779    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:14.427138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427110    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7297-cc3103"]
Oct 30 01:19:14.427745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427733    4577 cpu_manager.go:414] "containerMap: removing container" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy"
Oct 30 01:19:14.427802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427746    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy"
Oct 30 01:19:14.427802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427767    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy"
Oct 30 01:19:14.427802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427772    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy"
Oct 30 01:19:14.427802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427788    4577 cpu_manager.go:414] "containerMap: removing container" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy"
Oct 30 01:19:14.427802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427794    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427812    4577 cpu_manager.go:414] "containerMap: removing container" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427817    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427835    4577 cpu_manager.go:414] "containerMap: removing container" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427857    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427950    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="39fb2128-7e2f-410f-8a77-b4782ef926b7" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427960    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427968    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5d9c977-a868-48ad-86df-25432459b386" containerName="istio-proxy"
Oct 30 01:19:14.427986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.427975    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="333870ba-945f-4e5c-949a-ca525c615630" containerName="istio-proxy"
Oct 30 01:19:14.433354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.433337    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:14.447454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.447305    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7297-cc3103"]
Oct 30 01:19:14.456383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.456358    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4z6vl\""
Oct 30 01:19:14.456670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.456654    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 01:19:14.456949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.456908    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 01:19:14.570010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.569976    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96lpr\" (UniqueName: \"kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr\") pod \"test-curl-7297-cc3103\" (UID: \"cd2291a7-33be-40e2-b35f-39cd988f9102\") " pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:14.671816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.671771    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96lpr\" (UniqueName: \"kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr\") pod \"test-curl-7297-cc3103\" (UID: \"cd2291a7-33be-40e2-b35f-39cd988f9102\") " pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:14.697163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.697134    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96lpr\" (UniqueName: \"kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr\") pod \"test-curl-7297-cc3103\" (UID: \"cd2291a7-33be-40e2-b35f-39cd988f9102\") " pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:14.747529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:14.747504    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:15.055454 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:19:15.055424    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd2291a7_33be_40e2_b35f_39cd988f9102.slice/crio-f927d77ad800c89fd923bf49b5ff3a923e0628a7df85f5a6680022867bad2a32 WatchSource:0}: Error finding container f927d77ad800c89fd923bf49b5ff3a923e0628a7df85f5a6680022867bad2a32: Status 404 returned error can't find the container with id f927d77ad800c89fd923bf49b5ff3a923e0628a7df85f5a6680022867bad2a32
Oct 30 01:19:15.056522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:15.056493    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7297-cc3103"]
Oct 30 01:19:15.514705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:15.514664    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:15.627255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:15.626569    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7297-cc3103" event={"ID":"cd2291a7-33be-40e2-b35f-39cd988f9102","Type":"ContainerStarted","Data":"f927d77ad800c89fd923bf49b5ff3a923e0628a7df85f5a6680022867bad2a32"}
Oct 30 01:19:17.323031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:17.322992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:17.650185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:17.650147    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7297-cc3103" event={"ID":"cd2291a7-33be-40e2-b35f-39cd988f9102","Type":"ContainerStarted","Data":"e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202"}
Oct 30 01:19:17.695688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:17.695630    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7297-cc3103" podStartSLOduration=1.665695474 podStartE2EDuration="3.695614082s" totalImagesPullingTime="2.029918608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:17.692980298 +0000 UTC m=+2265.722290528" watchObservedRunningTime="2026-10-30 01:19:17.695614082 +0000 UTC m=+2265.724924312"
Oct 30 01:19:18.901727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:18.901690    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7297-cc3103"]
Oct 30 01:19:18.908701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:18.908671    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7297-cc3103"]
Oct 30 01:19:19.049082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.049044    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7297-1ecb85"]
Oct 30 01:19:19.050112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.050093    4577 cpu_manager.go:414] "containerMap: removing container" podUID="cd2291a7-33be-40e2-b35f-39cd988f9102" containerName="test-curl-7297-cc3103"
Oct 30 01:19:19.050229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.050113    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd2291a7-33be-40e2-b35f-39cd988f9102" containerName="test-curl-7297-cc3103"
Oct 30 01:19:19.050229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.050217    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="32b69ca6-e4e4-43ef-ab35-58f8d236a965" containerName="istio-proxy"
Oct 30 01:19:19.050229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.050225    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd2291a7-33be-40e2-b35f-39cd988f9102" containerName="test-curl-7297-cc3103"
Oct 30 01:19:19.053964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.053947    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:19.068379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.068350    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7297-1ecb85"]
Oct 30 01:19:19.172091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.172003    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4r4\" (UniqueName: \"kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4\") pod \"test-curl-7297-1ecb85\" (UID: \"53470495-e3cd-4e69-8ab4-8b75d809a05a\") " pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:19.273506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.273465    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4r4\" (UniqueName: \"kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4\") pod \"test-curl-7297-1ecb85\" (UID: \"53470495-e3cd-4e69-8ab4-8b75d809a05a\") " pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:19.298604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.298576    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4r4\" (UniqueName: \"kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4\") pod \"test-curl-7297-1ecb85\" (UID: \"53470495-e3cd-4e69-8ab4-8b75d809a05a\") " pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:19.368089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.368049    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:19.671187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.671153    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7297-cc3103" podUID="cd2291a7-33be-40e2-b35f-39cd988f9102" containerName="test-curl-7297-cc3103" containerID="cri-o://e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202" gracePeriod=2
Oct 30 01:19:19.673169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:19.673144    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7297-1ecb85"]
Oct 30 01:19:19.673753 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:19:19.673682    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53470495_e3cd_4e69_8ab4_8b75d809a05a.slice/crio-bab3366d162d383dfb7bd89f4823a2029d9de59b695b49eb942e0bfee895de3d WatchSource:0}: Error finding container bab3366d162d383dfb7bd89f4823a2029d9de59b695b49eb942e0bfee895de3d: Status 404 returned error can't find the container with id bab3366d162d383dfb7bd89f4823a2029d9de59b695b49eb942e0bfee895de3d
Oct 30 01:19:19.686474 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:19.686449    4577 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7297-cc3103\" is forbidden: User \"system:node:ip-10-0-141-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-17.ec2.internal' and this object" podUID="cd2291a7-33be-40e2-b35f-39cd988f9102" pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:20.681867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:20.681816    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7297-1ecb85" event={"ID":"53470495-e3cd-4e69-8ab4-8b75d809a05a","Type":"ContainerStarted","Data":"e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7"}
Oct 30 01:19:20.682386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:20.681895    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7297-1ecb85" event={"ID":"53470495-e3cd-4e69-8ab4-8b75d809a05a","Type":"ContainerStarted","Data":"bab3366d162d383dfb7bd89f4823a2029d9de59b695b49eb942e0bfee895de3d"}
Oct 30 01:19:20.724912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:20.724860    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7297-1ecb85" podStartSLOduration=1.724833855 podStartE2EDuration="1.724833855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:20.723582774 +0000 UTC m=+2268.752893006" watchObservedRunningTime="2026-10-30 01:19:20.724833855 +0000 UTC m=+2268.754144086"
Oct 30 01:19:20.904401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:20.904365    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:21.657201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:21.657161    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7297-1ecb85"]
Oct 30 01:19:21.661897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:21.661624    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7297-1ecb85"]
Oct 30 01:19:21.910743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:21.910682    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:21.956667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:21.956629    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr\" (UniqueName: \"kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr\") pod \"cd2291a7-33be-40e2-b35f-39cd988f9102\" (UID: \"cd2291a7-33be-40e2-b35f-39cd988f9102\") "
Oct 30 01:19:21.959023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:21.958994    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr" pod "cd2291a7-33be-40e2-b35f-39cd988f9102" (UID: "cd2291a7-33be-40e2-b35f-39cd988f9102"). InnerVolumeSpecName "kube-api-access-96lpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:22.058596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.058567    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96lpr\" (UniqueName: \"kubernetes.io/projected/cd2291a7-33be-40e2-b35f-39cd988f9102-kube-api-access-96lpr\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:22.113263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.113233    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd2291a7-33be-40e2-b35f-39cd988f9102" path="/var/lib/kubelet/pods/cd2291a7-33be-40e2-b35f-39cd988f9102/volumes"
Oct 30 01:19:22.241084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.241016    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:22.623880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.623779    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"]
Oct 30 01:19:22.625004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.624977    4577 cpu_manager.go:414] "containerMap: removing container" podUID="53470495-e3cd-4e69-8ab4-8b75d809a05a" containerName="test-curl-7297-1ecb85"
Oct 30 01:19:22.625004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.625004    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53470495-e3cd-4e69-8ab4-8b75d809a05a" containerName="test-curl-7297-1ecb85"
Oct 30 01:19:22.625318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.625290    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="53470495-e3cd-4e69-8ab4-8b75d809a05a" containerName="test-curl-7297-1ecb85"
Oct 30 01:19:22.630136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.630111    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.644577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.644551    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:22.648966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.648943    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-4107fc-openshift-default-dockercfg-7vw46\""
Oct 30 01:19:22.676282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.676253    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"]
Oct 30 01:19:22.709972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.709936    4577 generic.go:356] "Generic (PLEG): container finished" podID="cd2291a7-33be-40e2-b35f-39cd988f9102" containerID="e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202" exitCode=137
Oct 30 01:19:22.710133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.709987    4577 scope.go:122] "RemoveContainer" containerID="e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202"
Oct 30 01:19:22.710133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.710100    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7297-1ecb85" podUID="53470495-e3cd-4e69-8ab4-8b75d809a05a" containerName="test-curl-7297-1ecb85" containerID="cri-o://e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7" gracePeriod=2
Oct 30 01:19:22.710911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.710890    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-cc3103"
Oct 30 01:19:22.727645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.727613    4577 scope.go:122] "RemoveContainer" containerID="e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202"
Oct 30 01:19:22.727989 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:22.727968    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202\": container with ID starting with e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202 not found: ID does not exist" containerID="e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202"
Oct 30 01:19:22.728087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.728003    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202"} err="failed to get container status \"e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202\": rpc error: code = NotFound desc = could not find container \"e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202\": container with ID starting with e0a9de22bb81cec624f75525d8371c5b9e95de5d06725c084f01e63affe75202 not found: ID does not exist"
Oct 30 01:19:22.758077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.758046    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.792788    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.792906    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.792960    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.793023    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.793120    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.793159    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.793226    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.793261    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.798870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.793344    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ntvh\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.896753    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.896837    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.896963    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897067    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897095    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897171    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ntvh\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897276    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897361    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.901918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.897837    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.902598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.902232    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.902598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.902507    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.905866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.903531    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.927887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.918812    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.927887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.927166    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.940926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.940816    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.941818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.941745    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ntvh\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.946326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.946285    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo\") pod \"e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") " pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:22.980179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:22.980149    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:23.293092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.293052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:23.423133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.423106    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"]
Oct 30 01:19:23.423242 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:19:23.423135    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25ca204_e50f_4b0d_8cb7_b1b34a3fb4fb.slice/crio-67a3b3f93dca600e83b7d758cf93ceb055bc66d18cbc94c195f0f8c0c688e25f WatchSource:0}: Error finding container 67a3b3f93dca600e83b7d758cf93ceb055bc66d18cbc94c195f0f8c0c688e25f: Status 404 returned error can't find the container with id 67a3b3f93dca600e83b7d758cf93ceb055bc66d18cbc94c195f0f8c0c688e25f
Oct 30 01:19:23.425882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.425839    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:19:23.425990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.425915    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:19:23.425990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.425943    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:19:23.719363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.719261    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" event={"ID":"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb","Type":"ContainerStarted","Data":"7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"}
Oct 30 01:19:23.719363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.719327    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" event={"ID":"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb","Type":"ContainerStarted","Data":"67a3b3f93dca600e83b7d758cf93ceb055bc66d18cbc94c195f0f8c0c688e25f"}
Oct 30 01:19:23.772373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.772322    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podStartSLOduration=1.772307461 podStartE2EDuration="1.772307461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:23.770365259 +0000 UTC m=+2271.799675489" watchObservedRunningTime="2026-10-30 01:19:23.772307461 +0000 UTC m=+2271.801617691"
Oct 30 01:19:23.981639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:23.981607    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:24.611727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:24.611687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:24.939410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:24.939387    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:24.989068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:24.989033    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:25.057674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.057641    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4\" (UniqueName: \"kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4\") pod \"53470495-e3cd-4e69-8ab4-8b75d809a05a\" (UID: \"53470495-e3cd-4e69-8ab4-8b75d809a05a\") "
Oct 30 01:19:25.060169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.060141    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4" pod "53470495-e3cd-4e69-8ab4-8b75d809a05a" (UID: "53470495-e3cd-4e69-8ab4-8b75d809a05a"). InnerVolumeSpecName "kube-api-access-nq4r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:25.159890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.159786    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq4r4\" (UniqueName: \"kubernetes.io/projected/53470495-e3cd-4e69-8ab4-8b75d809a05a-kube-api-access-nq4r4\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:19:25.764707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.764674    4577 generic.go:356] "Generic (PLEG): container finished" podID="53470495-e3cd-4e69-8ab4-8b75d809a05a" containerID="e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7" exitCode=137
Oct 30 01:19:25.764957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.764826    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:25.767322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.766145    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:25.767322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.766642    4577 scope.go:122] "RemoveContainer" containerID="e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7"
Oct 30 01:19:25.767322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.767209    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:19:25.782080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.782059    4577 scope.go:122] "RemoveContainer" containerID="e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7"
Oct 30 01:19:25.782339 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:25.782316    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7\": container with ID starting with e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7 not found: ID does not exist" containerID="e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7"
Oct 30 01:19:25.782454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:25.782350    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7"} err="failed to get container status \"e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7\": rpc error: code = NotFound desc = could not find container \"e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7\": container with ID starting with e9e2a4cba990b6991d93ca6281fef1d6df75daa01aae85e7119eaa81392e37c7 not found: ID does not exist"
Oct 30 01:19:25.845389 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:25.845353    4577 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7297-1ecb85\" is forbidden: User \"system:node:ip-10-0-141-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-17.ec2.internal' and this object" podUID="53470495-e3cd-4e69-8ab4-8b75d809a05a" pod="opendatahub/test-curl-7297-1ecb85"
Oct 30 01:19:26.112681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:26.112602    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53470495-e3cd-4e69-8ab4-8b75d809a05a" path="/var/lib/kubelet/pods/53470495-e3cd-4e69-8ab4-8b75d809a05a/volumes"
Oct 30 01:19:26.169942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:26.169902    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:26.284133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:26.284096    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:26.518128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:26.518095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:27.014957 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:27.014921    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:19:27.845806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.845773    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:27.957028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.956997    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"]
Oct 30 01:19:27.961491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.961464    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:27.966682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.966656    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:27.982369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.982341    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"]
Oct 30 01:19:27.989925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.989890    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-4107fc-dockercfg-l24zb\""
Oct 30 01:19:27.990236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:27.990215    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-4107fc\""
Oct 30 01:19:28.003748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.003726    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.004071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.004038    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t9lq\" (UniqueName: \"kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq\") pod \"payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.020228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.020204    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"]
Oct 30 01:19:28.025909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.025839    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.041079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.041056    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"]
Oct 30 01:19:28.210923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.210378    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t9lq\" (UniqueName: \"kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq\") pod \"payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.210923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.210439    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.210923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.210533    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.210923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.210617    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9mdl\" (UniqueName: \"kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl\") pod \"payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.223306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.223271    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.440741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.438586    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9mdl\" (UniqueName: \"kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl\") pod \"payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.440741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.438875    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.441570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.441465    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.474858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.444517    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t9lq\" (UniqueName: \"kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq\") pod \"payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.474858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.470214    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:28.474858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.471500    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9mdl\" (UniqueName: \"kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl\") pod \"payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.474858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.472302    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:28.474858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.473540    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:28.802207 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:19:28.802179    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853ec6ff_84b8_4179_b14a_7e50f91534a9.slice/crio-ff4243e23d2cca178b1a4ebea8c802d56ad711527a1a3c558fe20629e125193e WatchSource:0}: Error finding container ff4243e23d2cca178b1a4ebea8c802d56ad711527a1a3c558fe20629e125193e: Status 404 returned error can't find the container with id ff4243e23d2cca178b1a4ebea8c802d56ad711527a1a3c558fe20629e125193e
Oct 30 01:19:28.802481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.802457    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"]
Oct 30 01:19:28.825565 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:19:28.825533    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec96526a_90f6_4096_a8e7_209f0740c279.slice/crio-d20cb8412a323c480ee09e3111a3fdfeef2af72a6adcc14feecefbb47960fe6d WatchSource:0}: Error finding container d20cb8412a323c480ee09e3111a3fdfeef2af72a6adcc14feecefbb47960fe6d: Status 404 returned error can't find the container with id d20cb8412a323c480ee09e3111a3fdfeef2af72a6adcc14feecefbb47960fe6d
Oct 30 01:19:28.826390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:28.826369    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"]
Oct 30 01:19:29.804685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.804647    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" event={"ID":"ec96526a-90f6-4096-a8e7-209f0740c279","Type":"ContainerStarted","Data":"e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7"}
Oct 30 01:19:29.805155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.804712    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" event={"ID":"ec96526a-90f6-4096-a8e7-209f0740c279","Type":"ContainerStarted","Data":"d20cb8412a323c480ee09e3111a3fdfeef2af72a6adcc14feecefbb47960fe6d"}
Oct 30 01:19:29.805155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.804752    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:29.806559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.806535    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" event={"ID":"853ec6ff-84b8-4179-b14a-7e50f91534a9","Type":"ContainerStarted","Data":"9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32"}
Oct 30 01:19:29.806711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.806573    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" event={"ID":"853ec6ff-84b8-4179-b14a-7e50f91534a9","Type":"ContainerStarted","Data":"ff4243e23d2cca178b1a4ebea8c802d56ad711527a1a3c558fe20629e125193e"}
Oct 30 01:19:29.806711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.806661    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:29.852557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.852516    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" podStartSLOduration=2.85250512 podStartE2EDuration="2.85250512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:29.848715398 +0000 UTC m=+2277.878025629" watchObservedRunningTime="2026-10-30 01:19:29.85250512 +0000 UTC m=+2277.881815351"
Oct 30 01:19:29.894216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:29.894155    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" podStartSLOduration=2.894137388 podStartE2EDuration="2.894137388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:29.891550265 +0000 UTC m=+2277.920860496" watchObservedRunningTime="2026-10-30 01:19:29.894137388 +0000 UTC m=+2277.923447620"
Oct 30 01:19:30.131583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:30.131489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:31.923785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:31.923746    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:33.720637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:33.720600    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:35.498838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:35.498807    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:37.214356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:37.214326    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:38.913954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:38.913908    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:40.702040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:40.702007    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:40.822645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:40.822618    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:19:40.826407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:40.825949    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:19:42.396476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:42.396430    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:44.220057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:44.220022    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:46.018202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:46.018163    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:48.022620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:48.022575    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:49.909217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:49.909186    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:51.570578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.570539    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"]
Oct 30 01:19:51.580691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.580669    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.597132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.597105    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"]
Oct 30 01:19:51.602905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.602878    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-4107fc-serving-cert\""
Oct 30 01:19:51.615080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.615049    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.615225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.615158    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.615225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.615212    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.615537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.615515    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55vnh\" (UniqueName: \"kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.658247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.658214    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:51.716894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.716863    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55vnh\" (UniqueName: \"kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.717091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.716984    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.717091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.717063    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.717218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.717110    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.717278 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:51.717259    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-serving-cert: secret "maas-api-e2e-shared-a-w4-4107fc-serving-cert" not found
Oct 30 01:19:51.717363 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:51.717350    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls podName:5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a nodeName:}" failed. No retries permitted until 2026-10-30 01:19:52.217328574 +0000 UTC m=+2300.246638804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls") pod "maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" (UID: "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a") : secret "maas-api-e2e-shared-a-w4-4107fc-serving-cert" not found
Oct 30 01:19:51.717725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.717704    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.719653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.719636    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.744002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.743964    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55vnh\" (UniqueName: \"kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:51.778616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.778580    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:51.894955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:51.893185    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:52.227956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:52.227826    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:52.230518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:52.230488    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:52.508546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:52.508509    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:52.901237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:52.901142    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"]
Oct 30 01:19:52.911716 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:19:52.911109    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b62f6b9_8dbb_4fa3_89a1_d76340b4b98a.slice/crio-b2206d78c6fb1e50b7b8f78e6b119445a641716c795d24dd8417dacc75d5dcde WatchSource:0}: Error finding container b2206d78c6fb1e50b7b8f78e6b119445a641716c795d24dd8417dacc75d5dcde: Status 404 returned error can't find the container with id b2206d78c6fb1e50b7b8f78e6b119445a641716c795d24dd8417dacc75d5dcde
Oct 30 01:19:53.045136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:53.045097    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" event={"ID":"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a","Type":"ContainerStarted","Data":"b2206d78c6fb1e50b7b8f78e6b119445a641716c795d24dd8417dacc75d5dcde"}
Oct 30 01:19:53.512531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:53.512496    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:54.055072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:54.055034    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" event={"ID":"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a","Type":"ContainerStarted","Data":"1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917"}
Oct 30 01:19:54.055537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:54.055141    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:19:54.106362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:54.106297    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" podStartSLOduration=2.693075666 podStartE2EDuration="3.106278109s" totalImagesPullingTime="413.202443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:54.102319005 +0000 UTC m=+2302.131629236" watchObservedRunningTime="2026-10-30 01:19:54.106278109 +0000 UTC m=+2302.135588340"
Oct 30 01:19:55.297630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:55.297592    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:57.016403 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:19:57.016363    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:19:57.051398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:57.051357    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:58.828640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:58.828601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:19:59.708885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.708837    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"]
Oct 30 01:19:59.709112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.709089    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy" containerID="cri-o://1fb0ed6d9098dcff8ec8a8a24bbdaf7a9d5d50198378581a30b288a00780172a" gracePeriod=30
Oct 30 01:19:59.722634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.722604    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"]
Oct 30 01:19:59.724003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.722875    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy" containerID="cri-o://672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d" gracePeriod=30
Oct 30 01:19:59.729925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.729881    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"]
Oct 30 01:19:59.730179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.730138    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" containerID="cri-o://d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8" gracePeriod=30
Oct 30 01:19:59.739139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.739108    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"]
Oct 30 01:19:59.739352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:19:59.739326    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy" containerID="cri-o://e641807a5d355611a8996147882b985ca1d680e22eee5c283f987db22a88525a" gracePeriod=30
Oct 30 01:20:00.070211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:00.070178    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:20:00.619788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:00.619749    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:02.399859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:02.399806    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:03.806597 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:03.806560    4577 patch_prober.go:29] interesting pod/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:20:03.807038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:03.806616    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:20:04.159380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:04.159297    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:05.042457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.042429    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.154399    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.157819    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.157928    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.157984    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.158070    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.158177    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.158229    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.158275    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.158373    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.158870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.158404    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy\") pod \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\" (UID: \"1acb033b-0c26-4221-bbd4-c927dd1f9bd5\") "
Oct 30 01:20:05.161411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.160700    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:05.161411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.160732    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.161667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.161643    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.161972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.161946    4577 generic.go:356] "Generic (PLEG): container finished" podID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerID="e641807a5d355611a8996147882b985ca1d680e22eee5c283f987db22a88525a" exitCode=0
Oct 30 01:20:05.162104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.162082    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.162178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.162112    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.162178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.162143    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" event={"ID":"b6a34547-fc81-4348-9ce8-a7aaafa6723f","Type":"ContainerDied","Data":"e641807a5d355611a8996147882b985ca1d680e22eee5c283f987db22a88525a"}
Oct 30 01:20:05.162665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.162639    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.162948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.162916    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.166393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.166362    4577 generic.go:356] "Generic (PLEG): container finished" podID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerID="1fb0ed6d9098dcff8ec8a8a24bbdaf7a9d5d50198378581a30b288a00780172a" exitCode=0
Oct 30 01:20:05.166534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.166492    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" event={"ID":"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0","Type":"ContainerDied","Data":"1fb0ed6d9098dcff8ec8a8a24bbdaf7a9d5d50198378581a30b288a00780172a"}
Oct 30 01:20:05.169013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.168985    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "kube-api-access-xlfmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.169986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.169948    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.176652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.171123    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:05.176950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.176820    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy" pod "1acb033b-0c26-4221-bbd4-c927dd1f9bd5" (UID: "1acb033b-0c26-4221-bbd4-c927dd1f9bd5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.177620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.177462    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:20:05.180685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.180289    4577 generic.go:356] "Generic (PLEG): container finished" podID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerID="d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8" exitCode=0
Oct 30 01:20:05.180685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.180450    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" event={"ID":"1acb033b-0c26-4221-bbd4-c927dd1f9bd5","Type":"ContainerDied","Data":"d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8"}
Oct 30 01:20:05.180685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.180520    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl" event={"ID":"1acb033b-0c26-4221-bbd4-c927dd1f9bd5","Type":"ContainerDied","Data":"fe985bbc3646268a2412303b4b64f8f9bc4838fcc4767776711037c8d7f7e2fb"}
Oct 30 01:20:05.180685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.180569    4577 scope.go:122] "RemoveContainer" containerID="d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8"
Oct 30 01:20:05.180685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.180569    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"
Oct 30 01:20:05.182352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.182148    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:20:05.184432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.184406    4577 generic.go:356] "Generic (PLEG): container finished" podID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerID="672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d" exitCode=0
Oct 30 01:20:05.184526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.184505    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" event={"ID":"48448591-cd0a-43a4-aacd-7aa72c32fdb0","Type":"ContainerDied","Data":"672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d"}
Oct 30 01:20:05.184628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.184550    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm" event={"ID":"48448591-cd0a-43a4-aacd-7aa72c32fdb0","Type":"ContainerDied","Data":"51af48237d1aa3cbe23b380246ad07da023537d72aadfd49283d9d558b0d3d69"}
Oct 30 01:20:05.184628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.184636    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"
Oct 30 01:20:05.196892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.196839    4577 scope.go:122] "RemoveContainer" containerID="d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8"
Oct 30 01:20:05.197256 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:20:05.197226    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8\": container with ID starting with d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8 not found: ID does not exist" containerID="d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8"
Oct 30 01:20:05.197382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.197263    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8"} err="failed to get container status \"d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8\": rpc error: code = NotFound desc = could not find container \"d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8\": container with ID starting with d5fd1abb1f1321fd0e92140d0c2df7dde1fd7744291162778dc8c07a946837a8 not found: ID does not exist"
Oct 30 01:20:05.197382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.197289    4577 scope.go:122] "RemoveContainer" containerID="672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d"
Oct 30 01:20:05.207677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.207658    4577 scope.go:122] "RemoveContainer" containerID="672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d"
Oct 30 01:20:05.208151 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:20:05.208127    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d\": container with ID starting with 672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d not found: ID does not exist" containerID="672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d"
Oct 30 01:20:05.208251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.208157    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d"} err="failed to get container status \"672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d\": rpc error: code = NotFound desc = could not find container \"672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d\": container with ID starting with 672eb17882670b5ac3f4a8fc06127c6c817a201093dd5a7881fb443dfbaf6a6d not found: ID does not exist"
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271134    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271201    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271239    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271283    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271311    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271348    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271409    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271445    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271530    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271578    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271628    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271662    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271705    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271750    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271783    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271823    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271868    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271916    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.271982    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.272433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.272028    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.273515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.272986    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.273515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273078    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.273515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273129    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.273515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273164    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert\") pod \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\" (UID: \"b6a34547-fc81-4348-9ce8-a7aaafa6723f\") "
Oct 30 01:20:05.273515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273355    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:05.273515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273462    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.273942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273676    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.273942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.273861    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.275627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.274666    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.275627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.275610    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.275829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.275798    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.276949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.276657    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.277054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.277030    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.277979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.277236    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.277979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.277437    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.277979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.277615    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.277979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.277808    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.278266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.278119    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:05.279040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.279002    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "kube-api-access-mbmk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.279190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.279118    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:05.279190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.279181    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.279434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.279241    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket\") pod \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\" (UID: \"48448591-cd0a-43a4-aacd-7aa72c32fdb0\") "
Oct 30 01:20:05.279434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.279333    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data\") pod \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\" (UID: \"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0\") "
Oct 30 01:20:05.280228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280210    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlfmf\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-kube-api-access-xlfmf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280236    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280252    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280264    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280279    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280294    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280308    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280320    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280334    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280347    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280361    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280374    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280388    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280401    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280414    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280455    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.280474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280469    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280484    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280500    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280517    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbmk5\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-kube-api-access-mbmk5\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280529    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280541    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280553    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb033b-0c26-4221-bbd4-c927dd1f9bd5-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280233    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.280768    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.281047    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:05.281359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.281145    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.281948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.281452    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:05.281948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.281602    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.282557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.282535    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.283021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.282997    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8" pod "48448591-cd0a-43a4-aacd-7aa72c32fdb0" (UID: "48448591-cd0a-43a4-aacd-7aa72c32fdb0"). InnerVolumeSpecName "kube-api-access-nhtj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.283087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.283041    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.283406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.283391    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy" pod "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" (UID: "04b2fa2d-e5b3-4ed9-925d-166567ec1ce0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:05.283608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.283585    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg" pod "b6a34547-fc81-4348-9ce8-a7aaafa6723f" (UID: "b6a34547-fc81-4348-9ce8-a7aaafa6723f"). InnerVolumeSpecName "kube-api-access-86jmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:05.320337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.320302    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"]
Oct 30 01:20:05.326499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.326471    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-frhfl"]
Oct 30 01:20:05.381959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.381925    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhtj8\" (UniqueName: \"kubernetes.io/projected/48448591-cd0a-43a4-aacd-7aa72c32fdb0-kube-api-access-nhtj8\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.381959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.381953    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.381959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.381965    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86jmg\" (UniqueName: \"kubernetes.io/projected/b6a34547-fc81-4348-9ce8-a7aaafa6723f-kube-api-access-86jmg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.381978    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.381991    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.381999    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.382010    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.382022    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48448591-cd0a-43a4-aacd-7aa72c32fdb0-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.382033    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.382047    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a34547-fc81-4348-9ce8-a7aaafa6723f-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.382240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.382059    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48448591-cd0a-43a4-aacd-7aa72c32fdb0-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:05.550397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.550286    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"]
Oct 30 01:20:05.554346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.554312    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-qz4fm"]
Oct 30 01:20:05.927580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:05.927491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:06.112052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.112019    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" path="/var/lib/kubelet/pods/1acb033b-0c26-4221-bbd4-c927dd1f9bd5/volumes"
Oct 30 01:20:06.112522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.112492    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" path="/var/lib/kubelet/pods/48448591-cd0a-43a4-aacd-7aa72c32fdb0/volumes"
Oct 30 01:20:06.194735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.194647    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"
Oct 30 01:20:06.194908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.194651    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh" event={"ID":"04b2fa2d-e5b3-4ed9-925d-166567ec1ce0","Type":"ContainerDied","Data":"83ced90f7a19c56a4e7e0f0d9a068b133b7b94e182c1a249097594d12733e54d"}
Oct 30 01:20:06.194908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.194788    4577 scope.go:122] "RemoveContainer" containerID="1fb0ed6d9098dcff8ec8a8a24bbdaf7a9d5d50198378581a30b288a00780172a"
Oct 30 01:20:06.198629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.198604    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9" event={"ID":"b6a34547-fc81-4348-9ce8-a7aaafa6723f","Type":"ContainerDied","Data":"47e254383ccca04ca373cc612419c4f42eac3c83878bb9fa25e333e0fa9d7ee4"}
Oct 30 01:20:06.198773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.198639    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"
Oct 30 01:20:06.206245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.206229    4577 scope.go:122] "RemoveContainer" containerID="e641807a5d355611a8996147882b985ca1d680e22eee5c283f987db22a88525a"
Oct 30 01:20:06.254882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.254812    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"]
Oct 30 01:20:06.270721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.270686    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-756j9"]
Oct 30 01:20:06.306780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.306748    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"]
Oct 30 01:20:06.310369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:06.310347    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-664c2b-openshift-default-8449b76bd5-tq7jh"]
Oct 30 01:20:07.699772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:07.699739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:08.112287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:08.112255    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" path="/var/lib/kubelet/pods/04b2fa2d-e5b3-4ed9-925d-166567ec1ce0/volumes"
Oct 30 01:20:08.112777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:08.112761    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" path="/var/lib/kubelet/pods/b6a34547-fc81-4348-9ce8-a7aaafa6723f/volumes"
Oct 30 01:20:09.413498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:09.413465    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:10.031123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:10.031095    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"]
Oct 30 01:20:10.044181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:10.044155    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5e54b5f8-vqs4g"]
Oct 30 01:20:10.112392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:10.112350    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64fe5aba-5ede-4535-9991-4285a8038904" path="/var/lib/kubelet/pods/64fe5aba-5ede-4535-9991-4285a8038904/volumes"
Oct 30 01:20:11.102291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:11.102256    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:12.805347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:12.805296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:14.625497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:14.625458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:16.355871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:16.355472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:18.202784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:18.202748    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:20.072867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:20.072769    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:21.890259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:21.890224    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:22.012376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.012339    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:22.460699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.460658    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:22.855562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.855416    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:22.973896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.965276    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"]
Oct 30 01:20:22.982969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.982940    4577 cpu_manager.go:414] "containerMap: removing container" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy"
Oct 30 01:20:22.983176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.983164    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy"
Oct 30 01:20:22.983272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.983263    4577 cpu_manager.go:414] "containerMap: removing container" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy"
Oct 30 01:20:22.983371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.983360    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy"
Oct 30 01:20:22.983508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.983500    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy"
Oct 30 01:20:22.983581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:22.983572    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy"
Oct 30 01:20:23.001952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.001923    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy"
Oct 30 01:20:23.001952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.001945    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy"
Oct 30 01:20:23.002210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.002120    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="04b2fa2d-e5b3-4ed9-925d-166567ec1ce0" containerName="istio-proxy"
Oct 30 01:20:23.002210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.002131    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="48448591-cd0a-43a4-aacd-7aa72c32fdb0" containerName="istio-proxy"
Oct 30 01:20:23.002210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.002150    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1acb033b-0c26-4221-bbd4-c927dd1f9bd5" containerName="istio-proxy"
Oct 30 01:20:23.002210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.002157    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6a34547-fc81-4348-9ce8-a7aaafa6723f" containerName="istio-proxy"
Oct 30 01:20:23.005473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.005431    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:23.007890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.007816    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"]
Oct 30 01:20:23.008012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.007986    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.018948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.018917    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.019275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019248    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019319    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019624    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019676    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019727    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmdf\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019819    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.019914    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.020426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.020065    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.024664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.024641    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-bc127f-openshift-default-dockercfg-q6vw7\""
Oct 30 01:20:23.123354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123262    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.123354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123330    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.123622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123466    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.123622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123542    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.123622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123574    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.123814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123699    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.123814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123746    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.124001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123833    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmdf\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.124001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123881    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.124001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123901    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.124208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.123970    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.128980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.128916    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.130694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.130667    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.137071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.135429    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.137071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.135519    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.137071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.136478    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.141664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.139489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:23.194777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.192307    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.223871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.222683    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmdf\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf\") pod \"e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") " pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.336871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.336158    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:23.790305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.790268    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"]
Oct 30 01:20:23.793559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.793526    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:20:23.793657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.793601    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:20:23.793657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.793637    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:20:23.816824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:23.816791    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:24.452380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:24.452282    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" event={"ID":"d72b6dbb-08cd-48b1-a0af-167f0cde3191","Type":"ContainerStarted","Data":"a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"}
Oct 30 01:20:24.452380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:24.452354    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" event={"ID":"d72b6dbb-08cd-48b1-a0af-167f0cde3191","Type":"ContainerStarted","Data":"381a256126320c22993e0019822241b8dd372b724ec96effaef8ebc04ca7b95a"}
Oct 30 01:20:24.496067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:24.490951    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"]
Oct 30 01:20:24.496067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:24.491182    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy" containerID="cri-o://e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e" gracePeriod=30
Oct 30 01:20:24.527931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:24.527867    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podStartSLOduration=2.5278326140000003 podStartE2EDuration="2.527832614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:20:24.516246593 +0000 UTC m=+2332.545556825" watchObservedRunningTime="2026-10-30 01:20:24.527832614 +0000 UTC m=+2332.557142845"
Oct 30 01:20:25.336787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.336740    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:25.344288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.344259    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:25.463136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.463101    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:25.464070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.464041    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:20:25.507985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.507950    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:25.750796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.750760    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"]
Oct 30 01:20:25.751034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:25.751010    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy" containerID="cri-o://c36112aee151ae3817ea669d8fef7c5b15a31e45671ccff7b62b0267a6498a65" gracePeriod=30
Oct 30 01:20:26.809018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:26.808977    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.219:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:20:26.809479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:26.809084    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.219:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:20:26.825749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:26.825710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:26.940726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:26.940687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:27.018700 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:20:27.018659    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:20:27.385343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:27.385303    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:28.502669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.502616    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:28.516813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.516778    4577 generic.go:356] "Generic (PLEG): container finished" podID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerID="7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3" exitCode=0
Oct 30 01:20:28.517018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.516866    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" event={"ID":"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb","Type":"ContainerDied","Data":"7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"}
Oct 30 01:20:28.517365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.517340    4577 scope.go:122] "RemoveContainer" containerID="7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"
Oct 30 01:20:28.519430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.519398    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:20:28.519563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.519472    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:20:28.519563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.519511    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:20:28.621540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:28.621506    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:29.213317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.213241    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:29.527056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.527026    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" event={"ID":"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb","Type":"ContainerStarted","Data":"31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d"}
Oct 30 01:20:29.823284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.823258    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:20:29.873637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.873601    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901275    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901390    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901534    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901584    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901615    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901689    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901790    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.901914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901897    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert\") pod \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\" (UID: \"ec3a7908-00b6-47ce-9eb9-7d06febf2f49\") "
Oct 30 01:20:29.902920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.901140    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:29.905942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.903252    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:29.905942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.903409    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:29.905942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.903754    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:29.906987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.906957    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:29.907426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.906657    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:29.908997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.908950    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:29.913121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.913099    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "kube-api-access-d9fff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:29.914376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.914342    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy" pod "ec3a7908-00b6-47ce-9eb9-7d06febf2f49" (UID: "ec3a7908-00b6-47ce-9eb9-7d06febf2f49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:29.983888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.983280    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:20:29.983888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.983426    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused" start-of-body=
Oct 30 01:20:29.983888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:29.983468    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused"
Oct 30 01:20:30.003879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003816    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9fff\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-kube-api-access-d9fff\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.003879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003864    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.003879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003875    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.003879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003883    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.003879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003891    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.004185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003899    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.004185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003906    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.004185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003914    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.004185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.003922    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec3a7908-00b6-47ce-9eb9-7d06febf2f49-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:30.536177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.536148    4577 generic.go:356] "Generic (PLEG): container finished" podID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerID="e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e" exitCode=0
Oct 30 01:20:30.536648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.536195    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"
Oct 30 01:20:30.536648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.536231    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" event={"ID":"ec3a7908-00b6-47ce-9eb9-7d06febf2f49","Type":"ContainerDied","Data":"e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e"}
Oct 30 01:20:30.536648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.536284    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7" event={"ID":"ec3a7908-00b6-47ce-9eb9-7d06febf2f49","Type":"ContainerDied","Data":"e2b5d91b3f576a0f9ced558b04eb1422636e2d569a0f9e77444066c4ebb2d026"}
Oct 30 01:20:30.536648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.536313    4577 scope.go:122] "RemoveContainer" containerID="e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e"
Oct 30 01:20:30.576695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.576669    4577 scope.go:122] "RemoveContainer" containerID="e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e"
Oct 30 01:20:30.577614 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:20:30.577575    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e\": container with ID starting with e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e not found: ID does not exist" containerID="e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e"
Oct 30 01:20:30.577751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.577621    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e"} err="failed to get container status \"e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e\": rpc error: code = NotFound desc = could not find container \"e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e\": container with ID starting with e7478ce0b4a6baa673314f1c234559c15e55b3877f7d27a8091993d96b1f0f3e not found: ID does not exist"
Oct 30 01:20:30.602339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.602283    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"]
Oct 30 01:20:30.613279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.613251    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-rpnq7"]
Oct 30 01:20:30.981431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.981391    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused" start-of-body=
Oct 30 01:20:30.981636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:30.981478    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused"
Oct 30 01:20:31.195612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.195494    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:31.552871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.545039    4577 generic.go:356] "Generic (PLEG): container finished" podID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerID="c36112aee151ae3817ea669d8fef7c5b15a31e45671ccff7b62b0267a6498a65" exitCode=0
Oct 30 01:20:31.561154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.553957    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" event={"ID":"eba32f0d-75b2-4d77-90c6-be3def381eab","Type":"ContainerDied","Data":"c36112aee151ae3817ea669d8fef7c5b15a31e45671ccff7b62b0267a6498a65"}
Oct 30 01:20:31.566869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.561456    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229" event={"ID":"eba32f0d-75b2-4d77-90c6-be3def381eab","Type":"ContainerDied","Data":"a1f31915cab2cb8b3086c327e979921d63b136c87628532d99e68467bc219925"}
Oct 30 01:20:31.567136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.567112    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1f31915cab2cb8b3086c327e979921d63b136c87628532d99e68467bc219925"
Oct 30 01:20:31.589555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.589521    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:20:31.634928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.634890    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.634964    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.634993    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635018    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635104    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635151    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635175    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:31.635347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635226    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635333    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635362    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket\") pod \"eba32f0d-75b2-4d77-90c6-be3def381eab\" (UID: \"eba32f0d-75b2-4d77-90c6-be3def381eab\") "
Oct 30 01:20:31.635578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635381    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:31.635578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635561    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:31.635578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635569    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:31.635822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.635755    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:31.636138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.636073    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.636138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.636102    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.636138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.636116    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba32f0d-75b2-4d77-90c6-be3def381eab-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.636611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.636191    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.636611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.636289    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.637949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.637926    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:31.638072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.638014    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "kube-api-access-27x9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:31.638072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.638007    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:31.638205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.638027    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token" pod "eba32f0d-75b2-4d77-90c6-be3def381eab" (UID: "eba32f0d-75b2-4d77-90c6-be3def381eab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:31.738007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.737974    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.738007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.738004    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27x9j\" (UniqueName: \"kubernetes.io/projected/eba32f0d-75b2-4d77-90c6-be3def381eab-kube-api-access-27x9j\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.738007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.738014    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.738343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.738023    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba32f0d-75b2-4d77-90c6-be3def381eab-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:31.981320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.981268    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused" start-of-body=
Oct 30 01:20:31.981516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:31.981338    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused"
Oct 30 01:20:32.112585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.112548    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" path="/var/lib/kubelet/pods/ec3a7908-00b6-47ce-9eb9-7d06febf2f49/volumes"
Oct 30 01:20:32.586271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.586173    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"
Oct 30 01:20:32.682948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.682881    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"]
Oct 30 01:20:32.686566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.686535    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-kz229"]
Oct 30 01:20:32.983494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.983463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:32.983770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.983534    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused" start-of-body=
Oct 30 01:20:32.983952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:32.983933    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused"
Oct 30 01:20:33.982064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:33.982016    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused" start-of-body=
Oct 30 01:20:33.982508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:33.982089    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.219:15021/healthz/ready\": dial tcp 10.134.0.219:15021: connect: connection refused"
Oct 30 01:20:34.113414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:34.113384    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" path="/var/lib/kubelet/pods/eba32f0d-75b2-4d77-90c6-be3def381eab/volumes"
Oct 30 01:20:34.668216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:34.668174    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:34.988564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:34.988539    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:20:34.989063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:34.989047    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:20:34.989892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:34.989824    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:20:36.414468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:36.414434    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:38.175435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:38.175389    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:39.869264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:39.869225    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:41.024955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:41.024927    4577 scope.go:122] "RemoveContainer" containerID="c36112aee151ae3817ea669d8fef7c5b15a31e45671ccff7b62b0267a6498a65"
Oct 30 01:20:41.702423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:41.702392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:43.540758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:43.540635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:43.900341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:43.900258    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:45.304860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:45.304807    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:46.655686 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:46.655639    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"]
Oct 30 01:20:46.656220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:46.655931    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy" containerID="cri-o://3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1" gracePeriod=30
Oct 30 01:20:47.053523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:47.053491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:48.814024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:48.813974    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:50.598575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:50.598546    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:51.932386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:51.932328    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:20:52.039214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039181    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.039480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039449    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:52.039480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039459    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039544    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039655    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039696    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039762    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039794    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039945    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.039972    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.040003    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket\") pod \"76329b3f-3902-43cc-83a6-a705ed42974c\" (UID: \"76329b3f-3902-43cc-83a6-a705ed42974c\") "
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.040241    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.040682    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.040700    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.040766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.040713    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.043829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.040929    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:52.043829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.041703    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:20:52.050046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.046203    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:52.050046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.046326    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:20:52.050046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.049323    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:52.053948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.052271    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k" pod "76329b3f-3902-43cc-83a6-a705ed42974c" (UID: "76329b3f-3902-43cc-83a6-a705ed42974c"). InnerVolumeSpecName "kube-api-access-sbz8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:52.141581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.141549    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.141581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.141579    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.141806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.141595    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76329b3f-3902-43cc-83a6-a705ed42974c-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.141806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.141604    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76329b3f-3902-43cc-83a6-a705ed42974c-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.141806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.141612    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbz8k\" (UniqueName: \"kubernetes.io/projected/76329b3f-3902-43cc-83a6-a705ed42974c-kube-api-access-sbz8k\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.141806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.141621    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76329b3f-3902-43cc-83a6-a705ed42974c-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:20:52.505271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.505238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:52.755912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.755796    4577 generic.go:356] "Generic (PLEG): container finished" podID="76329b3f-3902-43cc-83a6-a705ed42974c" containerID="3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1" exitCode=0
Oct 30 01:20:52.755912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.755876    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"
Oct 30 01:20:52.755912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.755897    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" event={"ID":"76329b3f-3902-43cc-83a6-a705ed42974c","Type":"ContainerDied","Data":"3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1"}
Oct 30 01:20:52.756198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.755956    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8" event={"ID":"76329b3f-3902-43cc-83a6-a705ed42974c","Type":"ContainerDied","Data":"010b570a876bbc32666e18074300385201631206b2b70d8ffa2ab004357afbca"}
Oct 30 01:20:52.756198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.755984    4577 scope.go:122] "RemoveContainer" containerID="3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1"
Oct 30 01:20:52.769143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.769117    4577 scope.go:122] "RemoveContainer" containerID="3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1"
Oct 30 01:20:52.769542 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:20:52.769511    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1\": container with ID starting with 3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1 not found: ID does not exist" containerID="3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1"
Oct 30 01:20:52.769677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.769553    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1"} err="failed to get container status \"3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1\": rpc error: code = NotFound desc = could not find container \"3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1\": container with ID starting with 3c40b0e393b97791d12834b983d8aa31df66fc38c07707d2c78d55ad20b38ef1 not found: ID does not exist"
Oct 30 01:20:52.832787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.832751    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"]
Oct 30 01:20:52.847553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:52.847524    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-nrlj8"]
Oct 30 01:20:54.094115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094082    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"]
Oct 30 01:20:54.094723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094709    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy"
Oct 30 01:20:54.094770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094724    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy"
Oct 30 01:20:54.094806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094771    4577 cpu_manager.go:414] "containerMap: removing container" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy"
Oct 30 01:20:54.094806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094776    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy"
Oct 30 01:20:54.094806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094791    4577 cpu_manager.go:414] "containerMap: removing container" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy"
Oct 30 01:20:54.094806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094797    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy"
Oct 30 01:20:54.094974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094900    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="eba32f0d-75b2-4d77-90c6-be3def381eab" containerName="istio-proxy"
Oct 30 01:20:54.094974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094909    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" containerName="istio-proxy"
Oct 30 01:20:54.094974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.094918    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec3a7908-00b6-47ce-9eb9-7d06febf2f49" containerName="istio-proxy"
Oct 30 01:20:54.100292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.100272    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.114421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.114393    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76329b3f-3902-43cc-83a6-a705ed42974c" path="/var/lib/kubelet/pods/76329b3f-3902-43cc-83a6-a705ed42974c/volumes"
Oct 30 01:20:54.118521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.118500    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"]
Oct 30 01:20:54.128139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.128105    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-bc127f-serving-cert\""
Oct 30 01:20:54.182103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.182075    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.182304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.182225    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.182371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.182303    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqfv\" (UniqueName: \"kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.182444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.182396    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.283410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.283378    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.283582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.283473    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqfv\" (UniqueName: \"kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.283582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.283524    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.283668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.283652    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.284364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.284278    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.286070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.286045    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.286195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.286133    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.320920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.320886    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqfv\" (UniqueName: \"kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv\") pod \"maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.341809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.341754    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:54.421744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.421643    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:54.467722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.467638    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:54.741120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.741097    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"]
Oct 30 01:20:54.741684 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:20:54.741661    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efeba0b_699f_4979_b51b_55ca71b66736.slice/crio-e6c0b92d82ec32fb93db927830f7d203ca2df16231a3e39b65d5f6d73b73004c WatchSource:0}: Error finding container e6c0b92d82ec32fb93db927830f7d203ca2df16231a3e39b65d5f6d73b73004c: Status 404 returned error can't find the container with id e6c0b92d82ec32fb93db927830f7d203ca2df16231a3e39b65d5f6d73b73004c
Oct 30 01:20:54.773407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:54.773369    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr" event={"ID":"9efeba0b-699f-4979-b51b-55ca71b66736","Type":"ContainerStarted","Data":"e6c0b92d82ec32fb93db927830f7d203ca2df16231a3e39b65d5f6d73b73004c"}
Oct 30 01:20:55.783708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:55.783673    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr" event={"ID":"9efeba0b-699f-4979-b51b-55ca71b66736","Type":"ContainerStarted","Data":"b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709"}
Oct 30 01:20:55.784261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:55.783761    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:20:55.836919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:55.836866    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr" podStartSLOduration=1.45083038 podStartE2EDuration="1.836836141s" totalImagesPullingTime="386.005761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:20:55.833163086 +0000 UTC m=+2363.862473320" watchObservedRunningTime="2026-10-30 01:20:55.836836141 +0000 UTC m=+2363.866146372"
Oct 30 01:20:56.303464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:56.303432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:57.016327 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:20:57.016293    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:20:58.126729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:58.126687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:20:59.916132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:20:59.916098    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:01.700868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:01.695977    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:01.797612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:01.797584    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:21:03.548355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:03.548319    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:04.703357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:04.703310    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:05.296603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:05.296566    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:07.035861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:07.035814    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:08.545405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:08.545373    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:08.952119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:08.952027    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:10.723519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:10.723478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:12.481267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:12.481230    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:14.251871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:14.251324    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:14.787661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:14.787622    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:15.397187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.397139    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"]
Oct 30 01:21:15.409506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.409479    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.425058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.425025    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"]
Oct 30 01:21:15.439419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.439381    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-499482-openshift-default-dockercfg-6fv68\""
Oct 30 01:21:15.505558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.501215    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4dl\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.505558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.501320    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.505558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.501385    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.505558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.501481    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.505558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.501611    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.505558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.501643    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.509517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.509463    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.509803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.509774    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.509992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.509974    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.526540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.526476    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.611779    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.611905    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612016    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612050    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612107    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612131    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612157    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612242    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4dl\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612299    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.612752    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.613461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.613453    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.614327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.613701    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.614327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.613961    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.614327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.614219    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.619827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.619757    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.619981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.619888    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.639886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.639820    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.640272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.640250    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4dl\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl\") pod \"e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") " pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:15.736664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.736617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:15.777212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:15.777115    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:16.202727 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:21:16.202670    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5087d495_29d9_43a8_872c_10a43a92f05d.slice/crio-983952a04d1d0e4346f1ea6078880854d39d37fcf0516f0768718d8d3a303866 WatchSource:0}: Error finding container 983952a04d1d0e4346f1ea6078880854d39d37fcf0516f0768718d8d3a303866: Status 404 returned error can't find the container with id 983952a04d1d0e4346f1ea6078880854d39d37fcf0516f0768718d8d3a303866
Oct 30 01:21:16.202929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:16.202818    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"]
Oct 30 01:21:16.205645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:16.205611    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:16.205762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:16.205692    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:16.205762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:16.205736    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:16.272959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:16.270409    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:17.002980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:17.002943    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" event={"ID":"5087d495-29d9-43a8-872c-10a43a92f05d","Type":"ContainerStarted","Data":"2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"}
Oct 30 01:21:17.003461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:17.003004    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" event={"ID":"5087d495-29d9-43a8-872c-10a43a92f05d","Type":"ContainerStarted","Data":"983952a04d1d0e4346f1ea6078880854d39d37fcf0516f0768718d8d3a303866"}
Oct 30 01:21:17.092878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:17.092809    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podStartSLOduration=2.09279374 podStartE2EDuration="2.09279374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:17.078392739 +0000 UTC m=+2385.107702971" watchObservedRunningTime="2026-10-30 01:21:17.09279374 +0000 UTC m=+2385.122103971"
Oct 30 01:21:17.777866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:17.777817    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:17.784361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:17.784332    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:18.011906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:18.011864    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:18.012989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:18.012971    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:21:18.128085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:18.128000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:19.584172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:19.584131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:19.761870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:19.759310    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:20.176196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:20.168665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:21.223586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:21.223551    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:21.345713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:21.345626    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:22.141699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:22.141657    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:22.868649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:22.868612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:23.845835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:23.845794    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:24.664264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:24.664227    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"]
Oct 30 01:21:24.664831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:24.664446    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy" containerID="cri-o://f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566" gracePeriod=30
Oct 30 01:21:25.086354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.086320    4577 generic.go:356] "Generic (PLEG): container finished" podID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerID="a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a" exitCode=0
Oct 30 01:21:25.086587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.086400    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" event={"ID":"d72b6dbb-08cd-48b1-a0af-167f0cde3191","Type":"ContainerDied","Data":"a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"}
Oct 30 01:21:25.086984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.086962    4577 scope.go:122] "RemoveContainer" containerID="a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"
Oct 30 01:21:25.089102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.089067    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:25.089189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.089160    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:25.089236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.089206    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:25.484867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.479942    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:21:25.635525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:25.635493    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:26.098471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:26.098434    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" event={"ID":"d72b6dbb-08cd-48b1-a0af-167f0cde3191","Type":"ContainerStarted","Data":"bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca"}
Oct 30 01:21:26.337012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:26.336978    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:21:26.338469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:26.338444    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused" start-of-body=
Oct 30 01:21:26.338617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:26.338501    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused"
Oct 30 01:21:27.013338 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:21:27.013291    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:21:27.337644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:27.337565    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused" start-of-body=
Oct 30 01:21:27.338043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:27.337645    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused"
Oct 30 01:21:27.612212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:27.612125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:27.898241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:27.898114    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:28.052870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:28.050793    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:28.337637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:28.337606    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused" start-of-body=
Oct 30 01:21:28.338115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:28.337659    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused"
Oct 30 01:21:28.437060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:28.437022    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:29.336949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:29.336907    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused" start-of-body=
Oct 30 01:21:29.337138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:29.336965    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused"
Oct 30 01:21:29.503795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:29.503763    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:29.976824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:29.976706    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:30.020442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.020418    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:21:30.097747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.097708    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:30.124523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124492    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124539    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124597    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124669    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124725    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124744    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:30.124988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124789    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124835    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124951    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.124988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.124963    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:30.125412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125005    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d\") pod \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\" (UID: \"ef6c8b03-8c63-4741-93f0-faa89aee5aac\") "
Oct 30 01:21:30.125412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125040    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:30.125412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125172    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:30.125412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125335    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:21:30.125813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125792    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.125813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125812    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.126005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125824    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.126005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125836    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.126005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.125862    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.128097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.128040    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:30.133907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.128336    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:21:30.133907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.128363    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "kube-api-access-7pl4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:30.133907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.128811    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token" pod "ef6c8b03-8c63-4741-93f0-faa89aee5aac" (UID: "ef6c8b03-8c63-4741-93f0-faa89aee5aac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:30.217775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.217737    4577 generic.go:356] "Generic (PLEG): container finished" podID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerID="f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566" exitCode=0
Oct 30 01:21:30.217941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.217805    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"
Oct 30 01:21:30.217941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.217826    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" event={"ID":"ef6c8b03-8c63-4741-93f0-faa89aee5aac","Type":"ContainerDied","Data":"f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566"}
Oct 30 01:21:30.217941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.217921    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77" event={"ID":"ef6c8b03-8c63-4741-93f0-faa89aee5aac","Type":"ContainerDied","Data":"80e0455817d3ab9ed0211f749eece42b1172e6e445a7052d282de12fa3ef70c1"}
Oct 30 01:21:30.218083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.217946    4577 scope.go:122] "RemoveContainer" containerID="f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566"
Oct 30 01:21:30.227903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.227821    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.227903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.227873    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pl4d\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-kube-api-access-7pl4d\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.227903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.227893    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.228094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.227925    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef6c8b03-8c63-4741-93f0-faa89aee5aac-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:30.231473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.231454    4577 scope.go:122] "RemoveContainer" containerID="f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566"
Oct 30 01:21:30.231799 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:21:30.231781    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566\": container with ID starting with f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566 not found: ID does not exist" containerID="f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566"
Oct 30 01:21:30.231959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.231806    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566"} err="failed to get container status \"f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566\": rpc error: code = NotFound desc = could not find container \"f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566\": container with ID starting with f0fdeb5f2ddf739f3d7bc261698dc7389105b49063ac698a69d88cb57fca9566 not found: ID does not exist"
Oct 30 01:21:30.296928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.296888    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"]
Oct 30 01:21:30.305059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.305033    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-wbj77"]
Oct 30 01:21:30.337792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.337760    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused" start-of-body=
Oct 30 01:21:30.337975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.337823    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused"
Oct 30 01:21:30.416892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.415652    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"]
Oct 30 01:21:30.416892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.416520    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy"
Oct 30 01:21:30.416892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.416538    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy"
Oct 30 01:21:30.416892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.416656    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" containerName="istio-proxy"
Oct 30 01:21:30.421702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.421679    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.434788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.434759    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"]
Oct 30 01:21:30.469911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.448203    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-a47b9a-openshift-default-dockercfg-s9ngk\""
Oct 30 01:21:30.490906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.490830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:30.538138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538106    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538281    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538356    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538393    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538436    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538460    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538487    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwb8x\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538556    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.538894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.538632    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.624370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.624338    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:30.640037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.639983    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640102    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640166    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640207    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640245    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640272    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640303    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwb8x\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640372    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640450    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640500    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.640883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640830    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.641052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.640943    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.641195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.641050    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.641195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.641061    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.642663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.642640    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.643152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.643136    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.678821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.678789    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.679902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.679875    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwb8x\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x\") pod \"e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") " pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:30.741468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.741254    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:30.747409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:30.747139    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:31.275705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.275664    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"]
Oct 30 01:21:31.278495 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:21:31.278466    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf506ab8c_31f5_449c_9662_90d953bcb38c.slice/crio-b951b879d7028f220d177a0bdb478a36069892fd9f2443bb16b4980bac21e79f WatchSource:0}: Error finding container b951b879d7028f220d177a0bdb478a36069892fd9f2443bb16b4980bac21e79f: Status 404 returned error can't find the container with id b951b879d7028f220d177a0bdb478a36069892fd9f2443bb16b4980bac21e79f
Oct 30 01:21:31.282172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.282135    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:31.282286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.282218    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:31.282286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.282260    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:31.337625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.337595    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused" start-of-body=
Oct 30 01:21:31.337748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.337653    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.223:15021/healthz/ready\": dial tcp 10.134.0.223:15021: connect: connection refused"
Oct 30 01:21:31.383504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:31.383458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:32.112021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.111990    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef6c8b03-8c63-4741-93f0-faa89aee5aac" path="/var/lib/kubelet/pods/ef6c8b03-8c63-4741-93f0-faa89aee5aac/volumes"
Oct 30 01:21:32.236559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.236530    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" event={"ID":"f506ab8c-31f5-449c-9662-90d953bcb38c","Type":"ContainerStarted","Data":"06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"}
Oct 30 01:21:32.236712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.236578    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" event={"ID":"f506ab8c-31f5-449c-9662-90d953bcb38c","Type":"ContainerStarted","Data":"b951b879d7028f220d177a0bdb478a36069892fd9f2443bb16b4980bac21e79f"}
Oct 30 01:21:32.342569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.342535    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:21:32.342979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.342955    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:21:32.343806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.343774    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:21:32.470921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.464226    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podStartSLOduration=2.464202298 podStartE2EDuration="2.464202298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:32.347127759 +0000 UTC m=+2400.376437990" watchObservedRunningTime="2026-10-30 01:21:32.464202298 +0000 UTC m=+2400.493512531"
Oct 30 01:21:32.748730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.748697    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:32.754861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:32.754820    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:33.126445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.126368    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"]
Oct 30 01:21:33.126888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.126571    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" containerID="cri-o://a64d05a4e28f6338120bfc31299a74245ac7073c245c58a75a48247f085eff0f" gracePeriod=30
Oct 30 01:21:33.248752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.248722    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:33.249683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.249658    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:21:33.465977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.465895    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:33.842158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.842119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:33.965196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:33.965156    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:34.780672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:34.780629    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:34.922884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:34.922835    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:35.582690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:35.582656    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:35.704971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:35.704938    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:35.943916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:35.891654    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:36.293012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:36.292968    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-2ksf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:21:36.293216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:36.293041    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:21:37.148600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:37.148563    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:37.266231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:37.266200    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:37.541680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:37.541635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:38.293097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.293059    4577 generic.go:356] "Generic (PLEG): container finished" podID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerID="a64d05a4e28f6338120bfc31299a74245ac7073c245c58a75a48247f085eff0f" exitCode=0
Oct 30 01:21:38.293485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.293137    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" event={"ID":"c5d44716-de3f-4435-a9ad-00da8719aee7","Type":"ContainerDied","Data":"a64d05a4e28f6338120bfc31299a74245ac7073c245c58a75a48247f085eff0f"}
Oct 30 01:21:38.422932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.422907    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 01:21:38.463458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463429    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463496    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463538    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463603    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463698    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463729    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463766    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463808    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.463926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463804    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:38.464170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.463981    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:38.464170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464003    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:38.464170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464085    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:38.464170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464090    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token\") pod \"c5d44716-de3f-4435-a9ad-00da8719aee7\" (UID: \"c5d44716-de3f-4435-a9ad-00da8719aee7\") "
Oct 30 01:21:38.464170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464162    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:21:38.464606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464587    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.464682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464608    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.464682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464617    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.464682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464628    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5d44716-de3f-4435-a9ad-00da8719aee7-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.464682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.464641    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.466189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.466164    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:38.466422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.466396    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:38.466543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.466490    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "kube-api-access-lt5bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:38.467007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.466970    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo" pod "c5d44716-de3f-4435-a9ad-00da8719aee7" (UID: "c5d44716-de3f-4435-a9ad-00da8719aee7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:21:38.565834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.565804    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.565834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.565831    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.566076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.565880    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt5bl\" (UniqueName: \"kubernetes.io/projected/c5d44716-de3f-4435-a9ad-00da8719aee7-kube-api-access-lt5bl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:38.566076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:38.565903    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5d44716-de3f-4435-a9ad-00da8719aee7-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:39.303072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:39.303038    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"
Oct 30 01:21:39.303072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:39.303053    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9" event={"ID":"c5d44716-de3f-4435-a9ad-00da8719aee7","Type":"ContainerDied","Data":"2da3e313ec3e2da1897bf86fe565ed270dda719ad4aafbd7ff33b81105866e6c"}
Oct 30 01:21:39.303611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:39.303127    4577 scope.go:122] "RemoveContainer" containerID="a64d05a4e28f6338120bfc31299a74245ac7073c245c58a75a48247f085eff0f"
Oct 30 01:21:39.386480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:39.386443    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"]
Oct 30 01:21:39.395419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:39.395395    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-2ksf9"]
Oct 30 01:21:39.599343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:39.599236    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:40.113837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:40.113807    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" path="/var/lib/kubelet/pods/c5d44716-de3f-4435-a9ad-00da8719aee7/volumes"
Oct 30 01:21:41.119879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:41.101670    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:41.153354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:41.151455    4577 scope.go:122] "RemoveContainer" containerID="3a277634ab3a3bc9e938e83bc4e1e6a180015e9f61c9832e09eafdc213e003f9"
Oct 30 01:21:41.451019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:41.450936    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:42.093499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:42.093468    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"]
Oct 30 01:21:42.150562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:42.150531    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:42.751009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:42.750980    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:43.138527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.138448    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"]
Oct 30 01:21:43.139521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.139498    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy"
Oct 30 01:21:43.139630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.139524    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy"
Oct 30 01:21:43.139709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.139697    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5d44716-de3f-4435-a9ad-00da8719aee7" containerName="istio-proxy"
Oct 30 01:21:43.145801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.145777    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.171573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.171527    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.172262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.172238    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.172485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.172466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cktvb\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.172690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.172636    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.172876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.172858    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.172984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.172971    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.173101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.173088    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.174554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.174531    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.174786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.174764    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.183901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.183882    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-637e08-openshift-default-dockercfg-7lspx\""
Oct 30 01:21:43.193040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.193016    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"]
Oct 30 01:21:43.223600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.223574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:43.275907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.275877    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.275922    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.275989    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276145    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cktvb\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276218    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276278    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276305    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276392    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276437    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276444    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276713    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276753    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.276994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.276949    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.277379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.277356    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.279205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.279180    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.279505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.279487    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.302857 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.302821    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cktvb\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.302965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.302954    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token\") pod \"e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") " pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.503873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.503351    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:43.520481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.519340    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:43.873325 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:21:43.873278    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb955911_3b23_422b_a79e_ebd4c34a4582.slice/crio-c7975e8885612b80bfc9049284fd6c809aea0988b73e923ccb594ffab0962029 WatchSource:0}: Error finding container c7975e8885612b80bfc9049284fd6c809aea0988b73e923ccb594ffab0962029: Status 404 returned error can't find the container with id c7975e8885612b80bfc9049284fd6c809aea0988b73e923ccb594ffab0962029
Oct 30 01:21:43.875704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.875647    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"]
Oct 30 01:21:43.877339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.877291    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:43.877461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.877383    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:43.877461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:43.877418    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:21:44.528656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:44.528620    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" event={"ID":"db955911-3b23-422b-a79e-ebd4c34a4582","Type":"ContainerStarted","Data":"f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"}
Oct 30 01:21:44.529144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:44.528684    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" event={"ID":"db955911-3b23-422b-a79e-ebd4c34a4582","Type":"ContainerStarted","Data":"c7975e8885612b80bfc9049284fd6c809aea0988b73e923ccb594ffab0962029"}
Oct 30 01:21:44.593524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:44.593454    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podStartSLOduration=1.5934367090000001 podStartE2EDuration="1.593436709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:44.59299851 +0000 UTC m=+2412.622308743" watchObservedRunningTime="2026-10-30 01:21:44.593436709 +0000 UTC m=+2412.622746942"
Oct 30 01:21:45.126961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:45.126930    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:45.508141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:45.508105    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:45.513794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:45.513770    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:45.536598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:45.536572    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:45.537456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:45.537435    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:21:46.935334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:46.935298    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:47.317301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:47.317265    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:48.685141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:48.685109    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:48.823412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:48.823382    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:49.041469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:49.041432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:50.766439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:50.766397    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:50.883367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:50.883322    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:51.000574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:51.000538    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:52.744522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:52.744487    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:54.718267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:54.718234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:56.027074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:56.027038    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:56.600287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:56.600252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:57.021082 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:21:57.021042    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:21:57.117015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.116970    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg" podUID="46854d92-0409-4d5d-aaf3-27316bc1838f" containerName="main" containerID="cri-o://9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5" gracePeriod=45
Oct 30 01:21:57.394587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.394560    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:21:57.530731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.530699    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir\") pod \"46854d92-0409-4d5d-aaf3-27316bc1838f\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") "
Oct 30 01:21:57.530972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.530770    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home\") pod \"46854d92-0409-4d5d-aaf3-27316bc1838f\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") "
Oct 30 01:21:57.530972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.530892    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs\" (UniqueName: \"kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs\") pod \"46854d92-0409-4d5d-aaf3-27316bc1838f\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") "
Oct 30 01:21:57.530972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.530949    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm\") pod \"46854d92-0409-4d5d-aaf3-27316bc1838f\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") "
Oct 30 01:21:57.531167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.530998    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f\" (UniqueName: \"kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f\") pod \"46854d92-0409-4d5d-aaf3-27316bc1838f\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") "
Oct 30 01:21:57.531167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531041    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache\") pod \"46854d92-0409-4d5d-aaf3-27316bc1838f\" (UID: \"46854d92-0409-4d5d-aaf3-27316bc1838f\") "
Oct 30 01:21:57.531167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531063    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir" pod "46854d92-0409-4d5d-aaf3-27316bc1838f" (UID: "46854d92-0409-4d5d-aaf3-27316bc1838f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:57.531167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531081    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home" pod "46854d92-0409-4d5d-aaf3-27316bc1838f" (UID: "46854d92-0409-4d5d-aaf3-27316bc1838f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:57.531385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531309    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache" pod "46854d92-0409-4d5d-aaf3-27316bc1838f" (UID: "46854d92-0409-4d5d-aaf3-27316bc1838f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:57.531748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531723    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:57.531820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531751    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:57.531820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.531765    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:57.533560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.533533    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f" pod "46854d92-0409-4d5d-aaf3-27316bc1838f" (UID: "46854d92-0409-4d5d-aaf3-27316bc1838f"). InnerVolumeSpecName "kube-api-access-djq6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:57.533560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.533554    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm" pod "46854d92-0409-4d5d-aaf3-27316bc1838f" (UID: "46854d92-0409-4d5d-aaf3-27316bc1838f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:57.533727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.533615    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs" pod "46854d92-0409-4d5d-aaf3-27316bc1838f" (UID: "46854d92-0409-4d5d-aaf3-27316bc1838f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:21:57.539331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.539311    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:57.633348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.633313    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46854d92-0409-4d5d-aaf3-27316bc1838f-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:57.633348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.633343    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46854d92-0409-4d5d-aaf3-27316bc1838f-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:57.633348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.633352    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djq6f\" (UniqueName: \"kubernetes.io/projected/46854d92-0409-4d5d-aaf3-27316bc1838f-kube-api-access-djq6f\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:21:57.690213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.690176    4577 generic.go:356] "Generic (PLEG): container finished" podID="46854d92-0409-4d5d-aaf3-27316bc1838f" containerID="9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5" exitCode=0
Oct 30 01:21:57.690401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.690231    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"
Oct 30 01:21:57.690401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.690259    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg" event={"ID":"46854d92-0409-4d5d-aaf3-27316bc1838f","Type":"ContainerDied","Data":"9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5"}
Oct 30 01:21:57.690401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.690304    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg" event={"ID":"46854d92-0409-4d5d-aaf3-27316bc1838f","Type":"ContainerDied","Data":"d279caa18a77c1da1bd1d961d19ebc184bbf53cef05d236f83d976c9ffba4af2"}
Oct 30 01:21:57.690401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.690320    4577 scope.go:122] "RemoveContainer" containerID="9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5"
Oct 30 01:21:57.702813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.702788    4577 scope.go:122] "RemoveContainer" containerID="9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5"
Oct 30 01:21:57.703101 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:21:57.703080    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5\": container with ID starting with 9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5 not found: ID does not exist" containerID="9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5"
Oct 30 01:21:57.703180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.703116    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5"} err="failed to get container status \"9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5\": rpc error: code = NotFound desc = could not find container \"9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5\": container with ID starting with 9ac3f0e3caa28bf1eed93da1d889fb6d3c348cb52db54a5c0931d19f987f84a5 not found: ID does not exist"
Oct 30 01:21:57.751322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.751297    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"]
Oct 30 01:21:57.757133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:57.757104    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/auth-test-model-9da49f-kserve-845b7c99bf-z9rcg"]
Oct 30 01:21:58.115340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:58.115301    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46854d92-0409-4d5d-aaf3-27316bc1838f" path="/var/lib/kubelet/pods/46854d92-0409-4d5d-aaf3-27316bc1838f/volumes"
Oct 30 01:21:58.280873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:58.241965    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:58.365953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:58.365290    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:21:58.505870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:21:58.503369    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:00.224672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:00.224635    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:02.044044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:02.043998    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:02.166382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:02.166351    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:03.715799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:03.715761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:03.833540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:03.833466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:03.944672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:03.944624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:05.666355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.666319    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"]
Oct 30 01:22:05.666972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.666958    4577 cpu_manager.go:414] "containerMap: removing container" podUID="46854d92-0409-4d5d-aaf3-27316bc1838f" containerName="main"
Oct 30 01:22:05.667021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.666972    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46854d92-0409-4d5d-aaf3-27316bc1838f" containerName="main"
Oct 30 01:22:05.667120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.667111    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="46854d92-0409-4d5d-aaf3-27316bc1838f" containerName="main"
Oct 30 01:22:05.672079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.672057    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.689386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.689361    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"]
Oct 30 01:22:05.701830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.701787    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-2f820e-serving-cert\""
Oct 30 01:22:05.778354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.778326    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hxz7\" (UniqueName: \"kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.778546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.778457    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.778643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.778623    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.778733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.778720    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.780498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.780473    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:05.879902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.879827    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.880256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.880233    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.890080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.890042    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hxz7\" (UniqueName: \"kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.890571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.890547    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.890861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.890814    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.902779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.902741    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.908886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.908819    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:05.926879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.926786    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.930899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.930828    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hxz7\" (UniqueName: \"kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7\") pod \"maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:05.986352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:05.986325    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:06.293381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:06.293358    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"]
Oct 30 01:22:06.293648 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:22:06.293624    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515d429f_3299_42c3_96e1_4cbe67ae6cbb.slice/crio-3f0e410d373cdade0ba1fd878622da8b56f615ef67d4360248843740ebae9658 WatchSource:0}: Error finding container 3f0e410d373cdade0ba1fd878622da8b56f615ef67d4360248843740ebae9658: Status 404 returned error can't find the container with id 3f0e410d373cdade0ba1fd878622da8b56f615ef67d4360248843740ebae9658
Oct 30 01:22:06.295024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:06.295008    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:22:06.865836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:06.865795    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m" event={"ID":"515d429f-3299-42c3-96e1-4cbe67ae6cbb","Type":"ContainerStarted","Data":"3f0e410d373cdade0ba1fd878622da8b56f615ef67d4360248843740ebae9658"}
Oct 30 01:22:07.511495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:07.511463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:07.878132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:07.878035    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m" event={"ID":"515d429f-3299-42c3-96e1-4cbe67ae6cbb","Type":"ContainerStarted","Data":"c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9"}
Oct 30 01:22:07.878132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:07.878118    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:07.925981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:07.925928    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m" podStartSLOduration=2.4859338109999998 podStartE2EDuration="2.925913444s" totalImagesPullingTime="439.979633ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:22:07.923703021 +0000 UTC m=+2435.953013252" watchObservedRunningTime="2026-10-30 01:22:07.925913444 +0000 UTC m=+2435.955223675"
Oct 30 01:22:09.454706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:09.454665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:09.580930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:09.580898    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:09.693647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:09.693611    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:11.366615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:11.366579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:13.215826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:13.215794    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:13.890367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:13.890338    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:22:14.908596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:14.908562    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:16.635364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:16.635325    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:18.337629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:18.337593    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:20.100374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:20.100345    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:21.782678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.782642    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"]
Oct 30 01:22:21.788202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.788180    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.806266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.806232    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"]
Oct 30 01:22:21.814832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.814793    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-637e08-serving-cert\""
Oct 30 01:22:21.884651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.884622    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.884823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.884679    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.884933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.884917    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9kdw\" (UniqueName: \"kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.885081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.885056    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.886190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.886163    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:21.986297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.986266    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.986504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.986333    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.986504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.986494    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9kdw\" (UniqueName: \"kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.986637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.986581    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.987292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.987264    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.989540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.989516    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.989675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.989563    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:21.998602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:21.998577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:22.036452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:22.036374    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9kdw\" (UniqueName: \"kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw\") pod \"maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:22.104769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:22.104735    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:22.411924 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:22:22.411895    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10ae2f7_b5da_475e_955a_88a72cc2cf70.slice/crio-8680d1c8d7f060eb34da8a1a1d39831074e2dbdd3d2e3a9a439d689c86a8000c WatchSource:0}: Error finding container 8680d1c8d7f060eb34da8a1a1d39831074e2dbdd3d2e3a9a439d689c86a8000c: Status 404 returned error can't find the container with id 8680d1c8d7f060eb34da8a1a1d39831074e2dbdd3d2e3a9a439d689c86a8000c
Oct 30 01:22:22.412109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:22.412087    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"]
Oct 30 01:22:22.986868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:22.986572    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:23.100426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:23.100340    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x" event={"ID":"d10ae2f7-b5da-475e-955a-88a72cc2cf70","Type":"ContainerStarted","Data":"00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d"}
Oct 30 01:22:23.100596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:23.100440    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x" event={"ID":"d10ae2f7-b5da-475e-955a-88a72cc2cf70","Type":"ContainerStarted","Data":"8680d1c8d7f060eb34da8a1a1d39831074e2dbdd3d2e3a9a439d689c86a8000c"}
Oct 30 01:22:23.100721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:23.100693    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:23.158549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:23.158496    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x" podStartSLOduration=1.825540775 podStartE2EDuration="2.158482174s" totalImagesPullingTime="332.941399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:22:23.154460629 +0000 UTC m=+2451.183770861" watchObservedRunningTime="2026-10-30 01:22:23.158482174 +0000 UTC m=+2451.187792405"
Oct 30 01:22:23.692498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:23.692461    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:25.707803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:25.707764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:27.014052 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:22:27.014015    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:22:27.583934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:27.583898    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:28.430447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:28.430408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:28.542806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:28.542775    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:29.116603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:29.116575    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:22:29.345871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:29.342651    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:30.895987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:30.895909    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:31.271429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:31.271342    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:32.236971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.236942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:32.661460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.661254    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"]
Oct 30 01:22:32.667423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.667399    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.702647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.702616    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-dd80cb-openshift-default-dockercfg-dqc97\""
Oct 30 01:22:32.703919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.703894    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"]
Oct 30 01:22:32.717622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.717579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:32.729425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729396    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m79r2\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729452    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729506    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729541    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729576    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729604    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729664    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729738    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.729895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.729879    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.833680    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.833737    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.833774    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.833800    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.833890    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.833959    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.834075    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.834149    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m79r2\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.834876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.834191    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.863874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.841401    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.863874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.841656    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.863874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.842376    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.863874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.842624    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.863874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.843153    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.869931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.869868    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.884921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.880535    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m79r2\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.884921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.881041    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:32.884921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:32.881239    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy\") pod \"e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") " pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:33.010245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.010071    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:33.105887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.105733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:33.223283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.223188    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:33.377871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.377824    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"]
Oct 30 01:22:33.379347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.379306    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:33.379481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.379402    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:33.379481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:33.379451    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:34.054016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:34.053978    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:22:34.054213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:34.054045    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:22:34.268939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:34.268897    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" event={"ID":"61823c04-7687-49d3-95ef-915421584fa0","Type":"ContainerStarted","Data":"158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"}
Oct 30 01:22:34.269149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:34.268977    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" event={"ID":"61823c04-7687-49d3-95ef-915421584fa0","Type":"ContainerStarted","Data":"d342f6cd073f80c4d6e77a18e24741ee04ce0480eebc764d2c8eb4ae4d0b460e"}
Oct 30 01:22:34.318090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:34.317996    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podStartSLOduration=2.317982423 podStartE2EDuration="2.317982423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:22:34.317749676 +0000 UTC m=+2462.347059907" watchObservedRunningTime="2026-10-30 01:22:34.317982423 +0000 UTC m=+2462.347292653"
Oct 30 01:22:34.910114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:34.910071    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:35.010559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:35.010520    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:35.016644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:35.016617    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:35.280261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:35.280226    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:35.281336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:35.281313    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:22:36.291744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.291660    4577 generic.go:356] "Generic (PLEG): container finished" podID="5087d495-29d9-43a8-872c-10a43a92f05d" containerID="2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d" exitCode=0
Oct 30 01:22:36.292295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.291740    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" event={"ID":"5087d495-29d9-43a8-872c-10a43a92f05d","Type":"ContainerDied","Data":"2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"}
Oct 30 01:22:36.292370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.292302    4577 scope.go:122] "RemoveContainer" containerID="2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"
Oct 30 01:22:36.294045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.294018    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:36.294158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.294081    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:36.294158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.294130    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:36.729388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.729360    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:36.871897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:36.871831    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:37.301197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:37.301156    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" event={"ID":"5087d495-29d9-43a8-872c-10a43a92f05d","Type":"ContainerStarted","Data":"61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607"}
Oct 30 01:22:37.778893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:37.778833    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:22:37.779237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:37.778981    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused" start-of-body=
Oct 30 01:22:37.779237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:37.779041    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused"
Oct 30 01:22:38.569169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:38.569119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:38.687825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:38.687761    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:38.778293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:38.778257    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused" start-of-body=
Oct 30 01:22:38.778496 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:38.778322    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused"
Oct 30 01:22:39.123978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:39.123928    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:39.276231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:39.266885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:39.778321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:39.778285    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused" start-of-body=
Oct 30 01:22:39.778756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:39.778355    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused"
Oct 30 01:22:40.778291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:40.778251    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused" start-of-body=
Oct 30 01:22:40.778509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:40.778321    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused"
Oct 30 01:22:41.079870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:41.079754    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:41.778114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:41.778076    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused" start-of-body=
Oct 30 01:22:41.778302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:41.778159    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused"
Oct 30 01:22:42.779149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:42.779051    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused" start-of-body=
Oct 30 01:22:42.779149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:42.779121    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.225:15021/healthz/ready\": dial tcp 10.134.0.225:15021: connect: connection refused"
Oct 30 01:22:42.943575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:42.943533    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:43.784169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:43.784141    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:22:43.784633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:43.784615    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:22:43.785357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:43.785335    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:22:44.721291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:44.721249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:46.546610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:46.546575    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:48.590069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:48.590030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:49.959959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:49.959922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:50.346733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:50.346690    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:51.224944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.224906    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:51.361923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.361884    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:51.613885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.613834    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"]
Oct 30 01:22:51.618494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.618470    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.637949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.637922    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-1afb3e-openshift-default-dockercfg-fzzvm\""
Oct 30 01:22:51.677516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.677481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:51.690304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.690275    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"]
Oct 30 01:22:51.706977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.706937    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707020    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707103    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r2qc\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707143    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707199    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707221    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707259    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707351    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.707592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.707380    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.854774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.854733    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r2qc\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.871983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.871901    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.872273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.872251    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.872403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.872388    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.872534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.872520    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.872707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.872695    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.872801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.872791    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.873074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.873057    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.873211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.873200    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.873832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.873809    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.931872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.874484    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.931872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.874722    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.931872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.874965    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.931872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.891548    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.990262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.990230    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r2qc\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.990781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.990730    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:51.996449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:51.996399    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:52.011536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.011473    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo\") pod \"e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") " pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:52.244429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.244404    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:52.313627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.313592    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:52.426702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.426545    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:52.608148 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:22:52.608117    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172b3d82_5c75_4c05_8f65_1af8ca1a277d.slice/crio-228e060f6ba01a06a0a8d0779e874e0da1dd704868cfad51ab9a17641d71b1b1 WatchSource:0}: Error finding container 228e060f6ba01a06a0a8d0779e874e0da1dd704868cfad51ab9a17641d71b1b1: Status 404 returned error can't find the container with id 228e060f6ba01a06a0a8d0779e874e0da1dd704868cfad51ab9a17641d71b1b1
Oct 30 01:22:52.608669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.608599    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"]
Oct 30 01:22:52.620974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.620931    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:52.621093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.621001    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:52.621093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.621029    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:22:52.667130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:52.667052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:53.230877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:53.230671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:53.536933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:53.535102    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" event={"ID":"172b3d82-5c75-4c05-8f65-1af8ca1a277d","Type":"ContainerStarted","Data":"813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"}
Oct 30 01:22:53.536933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:53.535179    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" event={"ID":"172b3d82-5c75-4c05-8f65-1af8ca1a277d","Type":"ContainerStarted","Data":"228e060f6ba01a06a0a8d0779e874e0da1dd704868cfad51ab9a17641d71b1b1"}
Oct 30 01:22:53.612126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:53.612087    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:53.619724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:53.619674    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podStartSLOduration=2.619661433 podStartE2EDuration="2.619661433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:22:53.616653347 +0000 UTC m=+2481.645963579" watchObservedRunningTime="2026-10-30 01:22:53.619661433 +0000 UTC m=+2481.648971663"
Oct 30 01:22:54.250993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:54.250914    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:54.274968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:54.265136    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:54.301878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:54.301823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:54.542794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:54.542712    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:54.543643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:54.543624    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:22:55.976576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:55.976482    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:56.095681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:56.095648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:56.234866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:56.233186    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:56.738386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:56.738347    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:56.876249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:56.876157    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:57.172555 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:22:57.172463    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:22:57.712500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:57.712464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:57.878067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:57.876013    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:58.217798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:58.216084    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:59.279700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:59.279665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:22:59.441510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:22:59.441478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:00.314462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:00.314433    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:02.306004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.305967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:02.878510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.878461    4577 generic.go:356] "Generic (PLEG): container finished" podID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerID="06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6" exitCode=0
Oct 30 01:23:02.878754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.878527    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" event={"ID":"f506ab8c-31f5-449c-9662-90d953bcb38c","Type":"ContainerDied","Data":"06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"}
Oct 30 01:23:02.879226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.879202    4577 scope.go:122] "RemoveContainer" containerID="06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"
Oct 30 01:23:02.881525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.881493    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:02.881604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.881583    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:02.881647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:02.881629    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:03.250044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:03.249954    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:23:03.907628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:03.907596    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" event={"ID":"f506ab8c-31f5-449c-9662-90d953bcb38c","Type":"ContainerStarted","Data":"c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd"}
Oct 30 01:23:04.209743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:04.209641    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:04.748236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:04.748205    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:23:04.750588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:04.750557    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused" start-of-body=
Oct 30 01:23:04.750883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:04.750829    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused"
Oct 30 01:23:05.767870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:05.763958    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused" start-of-body=
Oct 30 01:23:05.767870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:05.764022    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused"
Oct 30 01:23:06.106094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:06.106005    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:06.757303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:06.757264    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused" start-of-body=
Oct 30 01:23:06.757669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:06.757626    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused"
Oct 30 01:23:07.753872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:07.751985    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused" start-of-body=
Oct 30 01:23:07.753872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:07.752048    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused"
Oct 30 01:23:07.930878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:07.930809    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:08.751246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:08.751212    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:08.756939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:08.756356    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused" start-of-body=
Oct 30 01:23:08.756939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:08.756482    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.226:15021/healthz/ready\": dial tcp 10.134.0.226:15021: connect: connection refused"
Oct 30 01:23:09.753812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:09.753781    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:23:09.754310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:09.754288    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:23:09.755127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:09.755105    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:23:09.880761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:09.880660    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:10.003812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.003784    4577 generic.go:356] "Generic (PLEG): container finished" podID="db955911-3b23-422b-a79e-ebd4c34a4582" containerID="f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04" exitCode=0
Oct 30 01:23:10.004001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.003867    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" event={"ID":"db955911-3b23-422b-a79e-ebd4c34a4582","Type":"ContainerDied","Data":"f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"}
Oct 30 01:23:10.004526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.004508    4577 scope.go:122] "RemoveContainer" containerID="f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"
Oct 30 01:23:10.006216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.006189    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:10.006311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.006277    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:10.006354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.006310    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:10.218819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:10.218738    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:11.029891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:11.028528    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" event={"ID":"db955911-3b23-422b-a79e-ebd4c34a4582","Type":"ContainerStarted","Data":"2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998"}
Oct 30 01:23:11.511403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:11.511372    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:23:11.511810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:11.511706    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused" start-of-body=
Oct 30 01:23:11.512058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:11.512030    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused"
Oct 30 01:23:11.799353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:11.799259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:12.509066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:12.509029    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused" start-of-body=
Oct 30 01:23:12.509578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:12.509081    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused"
Oct 30 01:23:13.509405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:13.509366    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused" start-of-body=
Oct 30 01:23:13.509967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:13.509435    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused"
Oct 30 01:23:13.513995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:13.513971    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:14.508717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:14.508683    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused" start-of-body=
Oct 30 01:23:14.508945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:14.508737    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused"
Oct 30 01:23:15.277530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:15.277499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:15.508372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:15.508341    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused" start-of-body=
Oct 30 01:23:15.508569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:15.508395    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused"
Oct 30 01:23:15.537297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:15.537222    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:23:16.508519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:16.508473    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused" start-of-body=
Oct 30 01:23:16.508955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:16.508549    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.227:15021/healthz/ready\": dial tcp 10.134.0.227:15021: connect: connection refused"
Oct 30 01:23:17.122008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.121976    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"]
Oct 30 01:23:17.125978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.125950    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.159409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.159287    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-dd80cb-serving-cert\""
Oct 30 01:23:17.181956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.179666    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"]
Oct 30 01:23:17.234718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.234687    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.234982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.234958    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.235083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.234997    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rphzz\" (UniqueName: \"kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.235256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.235238    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.259113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.259081    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:17.336704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.336670    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.336951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.336784    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.337033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.337001    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.337284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.337255    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rphzz\" (UniqueName: \"kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.337876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.337835    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.339657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.339634    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.339728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.339657    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.392312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.392218    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:17.396868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.392709    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rphzz\" (UniqueName: \"kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz\") pod \"maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.439161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.439115    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:17.527056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.521154    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:23:17.527056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.522866    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:23:17.750641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:17.750592    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"]
Oct 30 01:23:17.751186 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:23:17.751155    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f086f0_79fa_412f_a322_3426ba607137.slice/crio-61c370220e2f5c131a7620fa266516ee9d353c791342c54a8d34225cefe08667 WatchSource:0}: Error finding container 61c370220e2f5c131a7620fa266516ee9d353c791342c54a8d34225cefe08667: Status 404 returned error can't find the container with id 61c370220e2f5c131a7620fa266516ee9d353c791342c54a8d34225cefe08667
Oct 30 01:23:18.131784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:18.131752    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" event={"ID":"62f086f0-79fa-412f-a322-3426ba607137","Type":"ContainerStarted","Data":"61c370220e2f5c131a7620fa266516ee9d353c791342c54a8d34225cefe08667"}
Oct 30 01:23:19.128206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:19.128170    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:19.140870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:19.140810    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" event={"ID":"62f086f0-79fa-412f-a322-3426ba607137","Type":"ContainerStarted","Data":"47fe6af196cb4f16690e259b690d7286edda4a2c070a8c028f3cc6dd43d82964"}
Oct 30 01:23:19.141016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:19.140901    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:19.215141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:19.215087    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" podStartSLOduration=1.867793279 podStartE2EDuration="2.215067335s" totalImagesPullingTime="347.274056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:23:19.194287165 +0000 UTC m=+2507.223597397" watchObservedRunningTime="2026-10-30 01:23:19.215067335 +0000 UTC m=+2507.244377566"
Oct 30 01:23:20.939313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:20.939276    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:22.748428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:22.748391    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:23.137254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:23.137181    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:23.275560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:23.275522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:23.833525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:23.833493    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:24.873084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:24.873043    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:25.035347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:25.035314    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:25.151253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:25.151165    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.232:8443/health\": dial tcp 10.134.0.232:8443: connect: connection refused"
Oct 30 01:23:25.564789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:25.564753    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:25.712960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:25.712924    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:26.822501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:26.822466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:27.014244 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:23:27.014206    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:23:28.384491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:28.384463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:28.609400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:28.609369    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:29.222746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:29.222668    4577 generic.go:356] "Generic (PLEG): container finished" podID="62f086f0-79fa-412f-a322-3426ba607137" containerID="47fe6af196cb4f16690e259b690d7286edda4a2c070a8c028f3cc6dd43d82964" exitCode=1
Oct 30 01:23:29.223003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:29.222823    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" event={"ID":"62f086f0-79fa-412f-a322-3426ba607137","Type":"ContainerDied","Data":"47fe6af196cb4f16690e259b690d7286edda4a2c070a8c028f3cc6dd43d82964"}
Oct 30 01:23:29.223416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:29.223380    4577 scope.go:122] "RemoveContainer" containerID="47fe6af196cb4f16690e259b690d7286edda4a2c070a8c028f3cc6dd43d82964"
Oct 30 01:23:29.252680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:29.252653    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:29.434044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:29.434016    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:30.231778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:30.231730    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" event={"ID":"62f086f0-79fa-412f-a322-3426ba607137","Type":"ContainerStarted","Data":"41caef17a0c3afe2e551b976f69a1c40f0f47268ac280c58b37cb9184b2ad617"}
Oct 30 01:23:30.231992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:30.231970    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:30.834630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:30.834600    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:30.969664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:30.969628    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:31.089374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:31.089278    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:32.628689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:32.628599    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:34.438538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:34.438054    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:36.208103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:36.208067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:36.337343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:36.337316    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:23:37.934581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:37.934547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:38.061883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:38.060607    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:38.174479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:38.174328    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:39.728279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.728246    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"]
Oct 30 01:23:39.733818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.733796    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.789898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.786944    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-ae1820-serving-cert\""
Oct 30 01:23:39.814241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.814210    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:39.818347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.818318    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"]
Oct 30 01:23:39.822188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.822159    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.822344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.822233    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.822344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.822336    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8xmb\" (UniqueName: \"kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.822459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.822420    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.923701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.923659    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8xmb\" (UniqueName: \"kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.923939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.923772    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.924010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.923943    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.924073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.924010    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.924221 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:23:39.924200    4577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-serving-cert: secret "maas-api-e2e-extmod-b-ae1820-serving-cert" not found
Oct 30 01:23:39.924307 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:23:39.924300    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls podName:55ceaeb2-4b48-44d1-8a1b-b4ca43008a53 nodeName:}" failed. No retries permitted until 2026-10-30 01:23:40.424280299 +0000 UTC m=+2528.453590508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls") pod "maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" (UID: "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53") : secret "maas-api-e2e-extmod-b-ae1820-serving-cert" not found
Oct 30 01:23:39.925518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.925491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:39.925707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.925642    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.927436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.927413    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:39.967611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:39.967575    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8xmb\" (UniqueName: \"kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:40.053925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:40.053548    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:40.431375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:40.431282    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:40.433954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:40.433929    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\") pod \"maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:40.687303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:40.687213    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:41.132378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:41.132344    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"]
Oct 30 01:23:41.132940 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:23:41.132912    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ceaeb2_4b48_44d1_8a1b_b4ca43008a53.slice/crio-5a49834136b7dc666e892eddb3c3d7c62f5927455ef705d451716059bb812d2c WatchSource:0}: Error finding container 5a49834136b7dc666e892eddb3c3d7c62f5927455ef705d451716059bb812d2c: Status 404 returned error can't find the container with id 5a49834136b7dc666e892eddb3c3d7c62f5927455ef705d451716059bb812d2c
Oct 30 01:23:41.506322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:41.506288    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" event={"ID":"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53","Type":"ContainerStarted","Data":"5a49834136b7dc666e892eddb3c3d7c62f5927455ef705d451716059bb812d2c"}
Oct 30 01:23:41.718148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:41.718111    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:42.515789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:42.515755    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" event={"ID":"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53","Type":"ContainerStarted","Data":"41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3"}
Oct 30 01:23:42.516211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:42.515990    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:42.569080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:42.569029    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" podStartSLOduration=3.203140013 podStartE2EDuration="3.569015268s" totalImagesPullingTime="365.875255ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:23:42.568717215 +0000 UTC m=+2530.598027447" watchObservedRunningTime="2026-10-30 01:23:42.569015268 +0000 UTC m=+2530.598325539"
Oct 30 01:23:43.470519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:43.470464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:45.166171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:45.166133    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:46.673277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:46.673240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:47.279940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.276472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:47.620130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.620100    4577 generic.go:356] "Generic (PLEG): container finished" podID="61823c04-7687-49d3-95ef-915421584fa0" containerID="158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d" exitCode=0
Oct 30 01:23:47.620345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.620175    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" event={"ID":"61823c04-7687-49d3-95ef-915421584fa0","Type":"ContainerDied","Data":"158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"}
Oct 30 01:23:47.620670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.620653    4577 scope.go:122] "RemoveContainer" containerID="158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"
Oct 30 01:23:47.622481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.622449    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:47.622608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.622537    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:47.622608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:47.622589    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:23:48.590527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:48.590498    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:23:48.630473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:48.630436    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" event={"ID":"61823c04-7687-49d3-95ef-915421584fa0","Type":"ContainerStarted","Data":"74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4"}
Oct 30 01:23:49.011173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:49.011137    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:23:49.012621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:49.012593    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused" start-of-body=
Oct 30 01:23:49.012759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:49.012657    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused"
Oct 30 01:23:49.034308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:49.034280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:50.011123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:50.011080    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused" start-of-body=
Oct 30 01:23:50.011540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:50.011151    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused"
Oct 30 01:23:50.281189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:50.281099    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:23:50.796152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:50.796107    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:51.011100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:51.011068    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused" start-of-body=
Oct 30 01:23:51.011284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:51.011120    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused"
Oct 30 01:23:52.011574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:52.011536    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused" start-of-body=
Oct 30 01:23:52.012042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:52.011598    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused"
Oct 30 01:23:52.560313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:52.560278    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:53.011451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:53.011413    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused" start-of-body=
Oct 30 01:23:53.011638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:53.011467    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.230:15021/healthz/ready\": dial tcp 10.134.0.230:15021: connect: connection refused"
Oct 30 01:23:54.055830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:54.055797    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:23:54.060780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:54.060751    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:23:54.335447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:54.335363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:56.112137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:56.112077    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:57.015522 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:23:57.015490    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:23:57.849329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:57.849272    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:57.964696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:57.964473    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:23:59.642869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:23:59.640315    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:01.403780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:01.403750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:03.204891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:03.204708    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:05.060973 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:05.060937    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:06.824990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:06.824952    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:08.147033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:08.147000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:08.619357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:08.619319    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:09.408330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:09.408294    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:09.564229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:09.564198    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:09.693828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:09.693744    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:09.978185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:09.978165    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:10.152030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.150128    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:10.159069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.159037    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": read tcp 10.134.0.2:42398->10.134.0.231:15021: read: connection reset by peer" start-of-body=
Oct 30 01:24:10.159202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.159122    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": read tcp 10.134.0.2:42398->10.134.0.231:15021: read: connection reset by peer"
Oct 30 01:24:10.293735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.293611    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:10.542306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.542277    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:10.867451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.867416    4577 generic.go:356] "Generic (PLEG): container finished" podID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerID="813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7" exitCode=0
Oct 30 01:24:10.867670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.867487    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" event={"ID":"172b3d82-5c75-4c05-8f65-1af8ca1a277d","Type":"ContainerDied","Data":"813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"}
Oct 30 01:24:10.867954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.867937    4577 scope.go:122] "RemoveContainer" containerID="813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"
Oct 30 01:24:10.870180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.870146    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:24:10.870296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.870238    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:24:10.870296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.870290    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:24:10.973084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:10.973056    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:11.091210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:11.091179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:11.880294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:11.880222    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" event={"ID":"172b3d82-5c75-4c05-8f65-1af8ca1a277d","Type":"ContainerStarted","Data":"06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c"}
Oct 30 01:24:12.249269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:12.249233    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:24:12.249963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:12.249584    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused" start-of-body=
Oct 30 01:24:12.250678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:12.250648    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused"
Oct 30 01:24:12.333303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:12.333265    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:12.474436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:12.474399    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:12.592358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:12.592237    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:13.245069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:13.245031    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused" start-of-body=
Oct 30 01:24:13.248867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:13.245648    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused"
Oct 30 01:24:14.268293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:14.268256    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused" start-of-body=
Oct 30 01:24:14.268962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:14.268924    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused"
Oct 30 01:24:14.396630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:14.396592    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:15.247486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:15.247448    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused" start-of-body=
Oct 30 01:24:15.247801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:15.247773    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused"
Oct 30 01:24:15.302955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:15.302915    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:15.414208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:15.414172    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:16.124709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:16.124678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:16.251282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:16.251237    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused" start-of-body=
Oct 30 01:24:16.251570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:16.251534    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused"
Oct 30 01:24:17.266871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:17.256341    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused" start-of-body=
Oct 30 01:24:17.266871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:17.256423    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.231:15021/healthz/ready\": dial tcp 10.134.0.231:15021: connect: connection refused"
Oct 30 01:24:17.924725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:17.924691    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:18.251109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:18.251079    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:24:18.251590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:18.251568    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:24:18.252290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:18.252271    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:24:19.713410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:19.713375    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:21.522052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:21.516425    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:23.240592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:23.240559    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:23.369837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:23.369796    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:25.032974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:25.032918    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:27.005329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:27.005287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:27.015236 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:27.015203    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:24:27.792577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:27.792542    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:28.902622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:28.902585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:29.017223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:29.017191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:29.190922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:29.190120    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:29.461017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:29.460929    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:29.615337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:29.615305    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:29.737061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:29.737023    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:30.843503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:30.843468    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:32.246062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:32.246033    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"]
Oct 30 01:24:32.246592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:32.246255    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x" podUID="d10ae2f7-b5da-475e-955a-88a72cc2cf70" containerName="maas-api" containerID="cri-o://00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d" gracePeriod=30
Oct 30 01:24:32.305189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:32.305106    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:32.831241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:32.831204    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:32.832265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:32.832240    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:24:32.949204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:32.949115    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:33.030872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.022051    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls\") pod \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") "
Oct 30 01:24:33.030872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.022301    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw\" (UniqueName: \"kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw\") pod \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") "
Oct 30 01:24:33.030872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.022390    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca\") pod \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") "
Oct 30 01:24:33.030872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.022492    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs\") pod \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\" (UID: \"d10ae2f7-b5da-475e-955a-88a72cc2cf70\") "
Oct 30 01:24:33.030872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.027623    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca" pod "d10ae2f7-b5da-475e-955a-88a72cc2cf70" (UID: "d10ae2f7-b5da-475e-955a-88a72cc2cf70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:33.049357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.049319    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw" pod "d10ae2f7-b5da-475e-955a-88a72cc2cf70" (UID: "d10ae2f7-b5da-475e-955a-88a72cc2cf70"). InnerVolumeSpecName "kube-api-access-x9kdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:33.052243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.052208    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs" pod "d10ae2f7-b5da-475e-955a-88a72cc2cf70" (UID: "d10ae2f7-b5da-475e-955a-88a72cc2cf70"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:33.052615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.052529    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls" pod "d10ae2f7-b5da-475e-955a-88a72cc2cf70" (UID: "d10ae2f7-b5da-475e-955a-88a72cc2cf70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:33.125126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.125089    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9kdw\" (UniqueName: \"kubernetes.io/projected/d10ae2f7-b5da-475e-955a-88a72cc2cf70-kube-api-access-x9kdw\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:33.125126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.125117    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d10ae2f7-b5da-475e-955a-88a72cc2cf70-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:33.125126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.125127    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:33.125126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.125135    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d10ae2f7-b5da-475e-955a-88a72cc2cf70-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:33.272166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.268969    4577 generic.go:356] "Generic (PLEG): container finished" podID="d10ae2f7-b5da-475e-955a-88a72cc2cf70" containerID="00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d" exitCode=0
Oct 30 01:24:33.272166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.269079    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x" event={"ID":"d10ae2f7-b5da-475e-955a-88a72cc2cf70","Type":"ContainerDied","Data":"00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d"}
Oct 30 01:24:33.272166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.269130    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x" event={"ID":"d10ae2f7-b5da-475e-955a-88a72cc2cf70","Type":"ContainerDied","Data":"8680d1c8d7f060eb34da8a1a1d39831074e2dbdd3d2e3a9a439d689c86a8000c"}
Oct 30 01:24:33.272166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.269160    4577 scope.go:122] "RemoveContainer" containerID="00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d"
Oct 30 01:24:33.272166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.269356    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"
Oct 30 01:24:33.323100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.322992    4577 scope.go:122] "RemoveContainer" containerID="00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d"
Oct 30 01:24:33.328061 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:33.327973    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d\": container with ID starting with 00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d not found: ID does not exist" containerID="00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d"
Oct 30 01:24:33.328061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.328019    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d"} err="failed to get container status \"00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d\": rpc error: code = NotFound desc = could not find container \"00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d\": container with ID starting with 00384282de2ce8e44c0a7ceb3baba11258fe1f6ceb536e587327128e56c7322d not found: ID does not exist"
Oct 30 01:24:33.387021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.386989    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"]
Oct 30 01:24:33.405870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.404815    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-637e08-6cc8fb58d9-gtz7x"]
Oct 30 01:24:33.428869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:33.427166    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:34.170869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:34.165286    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d10ae2f7-b5da-475e-955a-88a72cc2cf70" path="/var/lib/kubelet/pods/d10ae2f7-b5da-475e-955a-88a72cc2cf70/volumes"
Oct 30 01:24:35.147684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:35.147640    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:36.906264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:36.906223    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:38.293611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:38.293577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:38.774639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:38.774599    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:39.510453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:39.510411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:39.661671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:39.661633    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:39.857349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:39.857263    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:40.075516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:40.075480    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:40.256645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:40.256606    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:40.288727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:40.288693    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"]
Oct 30 01:24:40.288939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:40.288916    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy" containerID="cri-o://286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad" gracePeriod=30
Oct 30 01:24:40.433350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:40.433316    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:40.561594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:40.561514    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:41.293678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:41.293644    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:41.596365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:41.584252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:41.798869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:41.794725    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:42.887271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:42.887230    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:43.299941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:43.299892    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:43.445869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:43.445599    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:43.640072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:43.638412    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"]
Oct 30 01:24:43.640072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:43.638784    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" containerID="cri-o://2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998" gracePeriod=30
Oct 30 01:24:43.656024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:43.655800    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:43.942922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:43.933130    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:45.304661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.304625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:45.616218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.616195    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 01:24:45.679590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.679548    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.679764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.679635    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.679764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.679689    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.679970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.679806    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.679970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.679860    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.679970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.679929    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.680155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.680007    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:45.680155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.680019    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.680155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.680101    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.680309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.680261    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:45.680309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.680284    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy\") pod \"d338c3c4-97df-4b76-a32b-38af17edba0b\" (UID: \"d338c3c4-97df-4b76-a32b-38af17edba0b\") "
Oct 30 01:24:45.681201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.681164    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.681413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.681394    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.681581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.681209    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:45.681581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.681443    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:45.681729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.681699    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:45.691935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.691815    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:24:45.691935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.691884    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "kube-api-access-qv6fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:45.693938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.693906    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:45.693938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.693926    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token" pod "d338c3c4-97df-4b76-a32b-38af17edba0b" (UID: "d338c3c4-97df-4b76-a32b-38af17edba0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:45.782657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782617    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.782657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782657    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv6fx\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-kube-api-access-qv6fx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.782886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782673    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.782886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782687    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.782886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782700    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d338c3c4-97df-4b76-a32b-38af17edba0b-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.782886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782713    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:45.782886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:45.782725    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d338c3c4-97df-4b76-a32b-38af17edba0b-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:46.413906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.413790    4577 generic.go:356] "Generic (PLEG): container finished" podID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerID="286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad" exitCode=0
Oct 30 01:24:46.413906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.413868    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"
Oct 30 01:24:46.413906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.413881    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" event={"ID":"d338c3c4-97df-4b76-a32b-38af17edba0b","Type":"ContainerDied","Data":"286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad"}
Oct 30 01:24:46.414432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.413947    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md" event={"ID":"d338c3c4-97df-4b76-a32b-38af17edba0b","Type":"ContainerDied","Data":"8f533b754e33f66909e7762b9e61f34b6f9c5cf82c1e909fa2eac370917eb432"}
Oct 30 01:24:46.414432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.413971    4577 scope.go:122] "RemoveContainer" containerID="286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad"
Oct 30 01:24:46.425943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.425926    4577 scope.go:122] "RemoveContainer" containerID="21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"
Oct 30 01:24:46.435811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.435793    4577 scope.go:122] "RemoveContainer" containerID="286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad"
Oct 30 01:24:46.436106 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:46.436088    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad\": container with ID starting with 286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad not found: ID does not exist" containerID="286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad"
Oct 30 01:24:46.436157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.436114    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad"} err="failed to get container status \"286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad\": rpc error: code = NotFound desc = could not find container \"286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad\": container with ID starting with 286e3d255993020f4f7a33003227251016e4bb9500bfe4c03fbb1d1409156bad not found: ID does not exist"
Oct 30 01:24:46.436157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.436135    4577 scope.go:122] "RemoveContainer" containerID="21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"
Oct 30 01:24:46.436367 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:46.436348    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28\": container with ID starting with 21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28 not found: ID does not exist" containerID="21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"
Oct 30 01:24:46.436416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.436373    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28"} err="failed to get container status \"21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28\": rpc error: code = NotFound desc = could not find container \"21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28\": container with ID starting with 21399c12f3bd0295bbe15c70f40727081a9d5b2715f770a7692ef003f5c73b28 not found: ID does not exist"
Oct 30 01:24:46.487792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.487759    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"]
Oct 30 01:24:46.501557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:46.501530    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9bfe36-openshift-default-7c444f5fc5-6m9md"]
Oct 30 01:24:47.105135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:47.105104    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:47.524232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:47.524199    4577 patch_prober.go:29] interesting pod/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:24:47.524655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:47.524261    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:24:48.114470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:48.114433    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" path="/var/lib/kubelet/pods/d338c3c4-97df-4b76-a32b-38af17edba0b/volumes"
Oct 30 01:24:48.948650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:48.948624    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:24:49.023574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.023540    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.023781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.023611    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.023883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.023784    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.023957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.023884    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.023957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.023926    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.024076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024003    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.024076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024014    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:49.024212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024078    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.024212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024179    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.024325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024258    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo\") pod \"db955911-3b23-422b-a79e-ebd4c34a4582\" (UID: \"db955911-3b23-422b-a79e-ebd4c34a4582\") "
Oct 30 01:24:49.024475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024448    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:49.024832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024804    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:49.024956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.024911    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:49.025066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.025044    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:49.025359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.025339    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.025574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.025546    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.025699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.025684    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.025797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.025785    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db955911-3b23-422b-a79e-ebd4c34a4582-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.025913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.025901    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.027516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.027490    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:49.027998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.027970    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "kube-api-access-cktvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:49.028113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.028051    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:49.028220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.028200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo" pod "db955911-3b23-422b-a79e-ebd4c34a4582" (UID: "db955911-3b23-422b-a79e-ebd4c34a4582"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:24:49.030242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.030211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:49.127555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.127522    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db955911-3b23-422b-a79e-ebd4c34a4582-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.127555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.127553    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cktvb\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-kube-api-access-cktvb\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.127555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.127564    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db955911-3b23-422b-a79e-ebd4c34a4582-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.127833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.127573    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db955911-3b23-422b-a79e-ebd4c34a4582-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:24:49.439216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.439184    4577 generic.go:356] "Generic (PLEG): container finished" podID="db955911-3b23-422b-a79e-ebd4c34a4582" containerID="2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998" exitCode=0
Oct 30 01:24:49.439394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.439250    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"
Oct 30 01:24:49.439394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.439274    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" event={"ID":"db955911-3b23-422b-a79e-ebd4c34a4582","Type":"ContainerDied","Data":"2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998"}
Oct 30 01:24:49.439394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.439331    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z" event={"ID":"db955911-3b23-422b-a79e-ebd4c34a4582","Type":"ContainerDied","Data":"c7975e8885612b80bfc9049284fd6c809aea0988b73e923ccb594ffab0962029"}
Oct 30 01:24:49.439394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.439357    4577 scope.go:122] "RemoveContainer" containerID="2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998"
Oct 30 01:24:49.453011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.452992    4577 scope.go:122] "RemoveContainer" containerID="f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"
Oct 30 01:24:49.463629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.463607    4577 scope.go:122] "RemoveContainer" containerID="2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998"
Oct 30 01:24:49.464021 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:49.463995    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998\": container with ID starting with 2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998 not found: ID does not exist" containerID="2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998"
Oct 30 01:24:49.464107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.464034    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998"} err="failed to get container status \"2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998\": rpc error: code = NotFound desc = could not find container \"2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998\": container with ID starting with 2b59bf1b26126e37eb025369184d1257be713830242c6ef926cb4662286bf998 not found: ID does not exist"
Oct 30 01:24:49.464107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.464057    4577 scope.go:122] "RemoveContainer" containerID="f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"
Oct 30 01:24:49.464400 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:49.464362    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04\": container with ID starting with f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04 not found: ID does not exist" containerID="f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"
Oct 30 01:24:49.464459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.464411    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04"} err="failed to get container status \"f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04\": rpc error: code = NotFound desc = could not find container \"f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04\": container with ID starting with f47c8a23c0f2a2576fe2013027cb61eda40a85fbadedeb2edce9ce157eb55f04 not found: ID does not exist"
Oct 30 01:24:49.505504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.505472    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"]
Oct 30 01:24:49.519386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:49.519361    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-637e08-openshift-default-6bd7f44457-zdk4z"]
Oct 30 01:24:50.113177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:50.113141    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" path="/var/lib/kubelet/pods/db955911-3b23-422b-a79e-ebd4c34a4582/volumes"
Oct 30 01:24:50.816328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:50.816296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:52.685823 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:52.685782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:54.433226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:54.433194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:55.504675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:55.504638    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:56.224861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:56.224813    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:57.016356 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:24:57.016322    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:24:58.136866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:58.132999    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:24:59.921597 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:24:59.921560    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:00.738925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:00.738830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:02.028618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:02.028579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:03.777270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:03.777222    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:05.497777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.497741    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:05.915697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.915615    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"]
Oct 30 01:25:05.916674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916637    4577 cpu_manager.go:414] "containerMap: removing container" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy"
Oct 30 01:25:05.916674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916669    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy"
Oct 30 01:25:05.916903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916714    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy"
Oct 30 01:25:05.916903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916720    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy"
Oct 30 01:25:05.916903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916731    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d10ae2f7-b5da-475e-955a-88a72cc2cf70" containerName="maas-api"
Oct 30 01:25:05.916903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916736    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d10ae2f7-b5da-475e-955a-88a72cc2cf70" containerName="maas-api"
Oct 30 01:25:05.916903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916744    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy"
Oct 30 01:25:05.916903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916749    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy"
Oct 30 01:25:05.917211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916920    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy"
Oct 30 01:25:05.917211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916936    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy"
Oct 30 01:25:05.917211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916944    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d10ae2f7-b5da-475e-955a-88a72cc2cf70" containerName="maas-api"
Oct 30 01:25:05.917211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.916949    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d338c3c4-97df-4b76-a32b-38af17edba0b" containerName="istio-proxy"
Oct 30 01:25:05.921298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.921273    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:05.945042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.945007    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"]
Oct 30 01:25:05.958891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.957030    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-1cb829-openshift-default-dockercfg-gh8kg\""
Oct 30 01:25:05.982522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:05.982452    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.114417    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.114602    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.114774    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdj6c\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.114875    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.114917    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.115197    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.115386    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.115552    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.125871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.115618    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.153580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.153544    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:06.222882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.222771    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.223176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.223152    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.223346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.223330    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj6c\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.229979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.229945    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.230219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.230203    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.230412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.230390    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.230535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.230497    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.230581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.230557    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.230644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.230628    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.231193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.231165    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.231562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.231536    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.232124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.232098    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.232289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.232270    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.232826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.232801    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.246777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.246522    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.255427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.255400    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.258178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.258118    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdj6c\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.260087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.260058    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token\") pod \"e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") " pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.544296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.544264    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:06.868470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.868429    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"]
Oct 30 01:25:06.869642 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:25:06.869610    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1bb547_5123_41b1_8a31_86b0b33b4837.slice/crio-b9fe4069c4013b59db05929b1b4b9f1d3fafdafdbe80f02f89f84bef8adc1a1d WatchSource:0}: Error finding container b9fe4069c4013b59db05929b1b4b9f1d3fafdafdbe80f02f89f84bef8adc1a1d: Status 404 returned error can't find the container with id b9fe4069c4013b59db05929b1b4b9f1d3fafdafdbe80f02f89f84bef8adc1a1d
Oct 30 01:25:06.873142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.873108    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:25:06.873253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.873187    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:25:06.873253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:06.873231    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:25:07.332113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:07.332075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:07.704969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:07.704884    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" event={"ID":"da1bb547-5123-41b1-8a31-86b0b33b4837","Type":"ContainerStarted","Data":"e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"}
Oct 30 01:25:07.705366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:07.704974    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" event={"ID":"da1bb547-5123-41b1-8a31-86b0b33b4837","Type":"ContainerStarted","Data":"b9fe4069c4013b59db05929b1b4b9f1d3fafdafdbe80f02f89f84bef8adc1a1d"}
Oct 30 01:25:07.754589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:07.754526    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podStartSLOduration=2.754507436 podStartE2EDuration="2.754507436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:25:07.752136859 +0000 UTC m=+2615.781447090" watchObservedRunningTime="2026-10-30 01:25:07.754507436 +0000 UTC m=+2615.783817670"
Oct 30 01:25:08.544703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:08.544673    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:08.551537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:08.551508    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:08.712926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:08.712899    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:08.714000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:08.713966    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:25:09.134940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:09.134907    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:10.163501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:10.163468    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:10.295193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:10.295162    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:11.206260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:11.206217    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:11.218912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:11.218884    4577 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-17.ec2.internal" podUID="ab07c2e2299f7ce9c9ccf7ec1f988b83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 01:25:11.930578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:11.930527    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:12.058709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:12.058675    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:12.296872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:12.293149    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:12.497099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:12.497067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:12.656233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:12.656153    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:12.766243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:12.766215    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:13.263989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:13.263837    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:13.378320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:13.378270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:13.505322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:13.505273    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:14.443257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:14.443218    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:15.225184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:15.225156    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:17.217489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:17.217444    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:19.109656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:19.109625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:20.800479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:20.800441    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:22.578548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:22.578499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:23.350420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:23.350385    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:24.546695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:24.546657    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:26.224469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:26.224429    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:27.015579 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:25:27.015542    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:25:28.036542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:28.036512    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:29.826608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:29.826567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:31.702292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:31.702238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:33.410289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:33.410221    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:35.160402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:35.160362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:36.960504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:36.960463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:38.810717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:38.810683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:40.611627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:40.611583    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:42.409436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:42.409400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:42.600698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:42.600618    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:42.722296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:42.722206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:43.217606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:43.217568    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:44.218050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:44.217995    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:45.352454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:45.352403    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:46.133811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:46.133764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:47.772839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:47.772797    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:49.456165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:49.456130    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:49.579291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:49.579252    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:49.691451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:49.691419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:51.408006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:51.407967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:53.118472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:53.118433    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:53.228539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:53.228508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:53.345871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:53.345230    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:55.042409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:55.042378    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:55.883400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:55.883369    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:56.029335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:56.029287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:56.371246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:56.371216    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:56.533811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:56.533259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:57.034388 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:25:57.034345    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:25:57.316361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:57.316269    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:57.431729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:57.431694    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:58.741723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:58.741688    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:58.877867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:58.869452    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:25:59.143148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:25:59.143051    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:00.284570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:00.284530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:00.395665 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:00.395630    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:01.113756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:01.113699    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:02.434910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:02.434829    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:02.829367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:02.829331    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:04.521134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:04.521076    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:06.221998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:06.221961    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:08.016770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:08.016685    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:09.739165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:09.739108    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:11.537526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:11.537489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:13.308032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:13.308002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:15.033189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:15.033158    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:16.764292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:16.764261    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:18.073685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:18.073652    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:18.767113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:18.767072    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:20.600091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:20.600053    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:22.401153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:22.401121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:23.473735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:23.473695    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:24.222198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:24.222150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:25.983951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:25.983919    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:27.076058 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:26:27.075989    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:26:27.823379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:27.823342    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:29.611053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:29.611014    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:31.399984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:31.399946    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:32.704654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:32.704565    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:32.884872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:32.878710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:33.204454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:33.204422    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:35.149813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:35.149784    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:36.917609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:36.917573    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:38.823293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:38.823255    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:39.203966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.202911    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.234:15021/healthz/ready\": read tcp 10.134.0.2:40758->10.134.0.234:15021: read: connection reset by peer" start-of-body=
Oct 30 01:26:39.203966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.202984    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.234:15021/healthz/ready\": read tcp 10.134.0.2:40758->10.134.0.234:15021: read: connection reset by peer"
Oct 30 01:26:39.482945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.482912    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:39.600462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.600402    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:39.786800 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.786713    4577 generic.go:356] "Generic (PLEG): container finished" podID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerID="e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9" exitCode=0
Oct 30 01:26:39.786979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.786791    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" event={"ID":"da1bb547-5123-41b1-8a31-86b0b33b4837","Type":"ContainerDied","Data":"e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"}
Oct 30 01:26:39.787402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.787382    4577 scope.go:122] "RemoveContainer" containerID="e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"
Oct 30 01:26:39.789144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.789119    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:26:39.789228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.789216    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:26:39.789275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:39.789251    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:26:40.695590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:40.695555    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:40.799484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:40.799447    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" event={"ID":"da1bb547-5123-41b1-8a31-86b0b33b4837","Type":"ContainerStarted","Data":"c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3"}
Oct 30 01:26:41.544377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:41.544344    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:26:41.544651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:41.544627    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused" start-of-body=
Oct 30 01:26:41.544765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:41.544684    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused"
Oct 30 01:26:42.012885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:42.012830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:42.559876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:42.554201    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused" start-of-body=
Oct 30 01:26:42.559876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:42.554259    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused"
Oct 30 01:26:42.713119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:42.713075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:43.550192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:43.550156    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused" start-of-body=
Oct 30 01:26:43.550805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:43.550772    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused"
Oct 30 01:26:44.504562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:44.504529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:44.558295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:44.558252    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused" start-of-body=
Oct 30 01:26:44.571063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:44.571012    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused"
Oct 30 01:26:45.563697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:45.563232    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused" start-of-body=
Oct 30 01:26:45.563697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:45.563309    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.234:15021/healthz/ready\": dial tcp 10.134.0.234:15021: connect: connection refused"
Oct 30 01:26:46.231043 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:46.231013    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:46.552320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:46.552242    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:26:46.553069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:46.553024    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:26:46.554001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:46.553983    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:26:48.024345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:48.024307    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:49.938835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:49.938795    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:51.721209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:51.721179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:53.305362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.305325    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:53.554151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.554118    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:53.702734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.702645    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"]
Oct 30 01:26:53.703434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.703417    4577 cpu_manager.go:414] "containerMap: removing container" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy"
Oct 30 01:26:53.703542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.703435    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy"
Oct 30 01:26:53.703603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.703568    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="db955911-3b23-422b-a79e-ebd4c34a4582" containerName="istio-proxy"
Oct 30 01:26:53.706428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.706408    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.731386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.731355    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"]
Oct 30 01:26:53.739336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.739308    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-c48628-openshift-default-dockercfg-nrrlw\""
Oct 30 01:26:53.756646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756615    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.756981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756659    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.756981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756702    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99ds6\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.756981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756731    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.756981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756781    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.756981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756928    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.756981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.756975    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.757618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.757083    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.757618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.757243    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.762385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.762357    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.859791    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.859869    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.859914    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99ds6\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.859943    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.859997    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.860134    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.860181    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.860294    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.860401    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.860952    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.861222    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.862468    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.867875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.863094    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.870553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.870517    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.927602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.927507    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99ds6\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.931111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.928311    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.962277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.934567    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.964466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.964431    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo\") pod \"e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") " pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:53.973189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:53.964923    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:54.024406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.024327    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:54.401034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.401002    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"]
Oct 30 01:26:54.402301 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:26:54.402273    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6ec37d_8703_4491_a295_91eb0e031794.slice/crio-4fd72329dbfc78629b7323c0ff1f660b7a1d90935689eea87cc51efcf2eab75c WatchSource:0}: Error finding container 4fd72329dbfc78629b7323c0ff1f660b7a1d90935689eea87cc51efcf2eab75c: Status 404 returned error can't find the container with id 4fd72329dbfc78629b7323c0ff1f660b7a1d90935689eea87cc51efcf2eab75c
Oct 30 01:26:54.405184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.405151    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:26:54.405280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.405225    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:26:54.405280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.405275    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:26:54.991276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.991238    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" event={"ID":"aa6ec37d-8703-4491-a295-91eb0e031794","Type":"ContainerStarted","Data":"e0780bb0f3158782707471cc027dc31d76daf852dcf4aa79ec15fbce11d7ce2c"}
Oct 30 01:26:54.991508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:54.991305    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" event={"ID":"aa6ec37d-8703-4491-a295-91eb0e031794","Type":"ContainerStarted","Data":"4fd72329dbfc78629b7323c0ff1f660b7a1d90935689eea87cc51efcf2eab75c"}
Oct 30 01:26:55.024664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:55.024636    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:55.031867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:55.031826    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:55.077005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:55.076959    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podStartSLOduration=2.07694452 podStartE2EDuration="2.07694452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:26:55.06143317 +0000 UTC m=+2723.090743400" watchObservedRunningTime="2026-10-30 01:26:55.07694452 +0000 UTC m=+2723.106254750"
Oct 30 01:26:55.374102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:55.374011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:55.998830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:55.998791    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:55.999834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:55.999815    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:26:57.017433 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:26:57.017397    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:26:57.173095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:57.173061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:26:58.979864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:26:58.979811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:00.831765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:00.831727    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:01.462801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:01.462753    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:01.574560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:01.574530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:02.084062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:02.084034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:02.208698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:02.208665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:02.331640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:02.331601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:02.447290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:02.447204    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:02.760705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:02.760663    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:02.877873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:02.877337    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:04.576857 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:04.576811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:06.350362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:06.350331    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:07.533284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:07.533245    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:07.671635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:07.671602    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:08.364370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:08.364330    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:09.238656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:09.238624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:09.437890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:09.435248    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.021815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.021770    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.136695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.136665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.255592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.255559    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.372108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.371998    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.595021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.594992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.708781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.708701    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:10.879175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:10.879119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:12.158890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:12.158795    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:13.914527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:13.914447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:15.490654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:15.490620    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:15.605603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:15.605569    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:16.002873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:16.002801    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:16.545447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:16.545396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:17.104866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:17.104794    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:17.288677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:17.288639    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:17.424273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:17.424153    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:17.563931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:17.563896    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:17.723996 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:17.723867    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:17.931629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:17.931587    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:18.057650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:18.057609    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:18.392249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:18.392150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:18.535324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:18.535278    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:18.662958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:18.662830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:18.846007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:18.845972    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:19.867055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:19.867021    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:20.005867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:20.001695    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:21.034720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:21.034686    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:21.218777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:21.218747    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:21.329181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:21.329103    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:21.521872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:21.519410    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:21.962460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:21.961413    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:23.591741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:23.591710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:23.716236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:23.716208    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:25.393731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:25.393688    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:27.015195 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:27:27.015160    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:27:27.232361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:27.232325    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:28.984355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:28.984319    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:29.257931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:29.254311    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:30.920083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:30.920043    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:32.382797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:32.382759    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:32.494204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:32.494160    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:32.935284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:32.935154    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:34.833508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:34.833467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:36.619794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:36.619764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:38.372668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:38.372612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:39.275218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:39.275183    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:39.900441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:39.900408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:40.015492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:40.015432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:40.255358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:40.255319    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:41.125053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:41.125021    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:42.080921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:42.080881    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:43.781577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:43.781542    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:45.684802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:45.684767    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:47.602169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:47.602125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:49.518474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:49.518431    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:49.628095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:49.628066    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:51.305936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:51.305885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:52.405030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:52.404997    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:52.517878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:52.517833    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:52.724142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:52.724030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:53.119137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:53.119098    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:53.259155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:53.259095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:53.401920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:53.393598    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:53.567871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:53.565738    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:53.678284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:53.678200    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:53.796206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:53.796173    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:54.031804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:54.031762    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:54.497009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:54.496970    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:54.607097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:54.607059    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:55.347934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:55.347899    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:55.469448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:55.469394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:56.393829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:56.393794    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:56.536500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:56.536458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:56.888598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:56.888565    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"]
Oct 30 01:27:56.893104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:56.893082    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:56.917780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:56.917734    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"]
Oct 30 01:27:57.121708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.098076    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1afb3e\"/\"openshift-service-ca.crt\""
Oct 30 01:27:57.121708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.098171    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-1afb3e\"/\"agc-cred-1aa3b9ce7f6d\""
Oct 30 01:27:57.121708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.098258    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-1afb3e\"/\"payload-processing-external-model-e2e--8ac1e578-dockercfg-wt9sb\""
Oct 30 01:27:57.121708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.098287    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1afb3e\"/\"kube-root-ca.crt\""
Oct 30 01:27:57.121708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.098323    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1afb3e\"/\"routing-overlay\""
Oct 30 01:27:57.121708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.098409    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-1afb3e\"/\"payload-processing-external-model-plugins-e2e-extmod-b-1afb3e\""
Oct 30 01:27:57.166064 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:27:57.165971    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:27:57.201445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.201400    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjfd7\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.201632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.201586    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.201632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.201629    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.201742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.201653    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.303764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.303728    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.304007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.303793    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.304007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.303915    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.304247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.304228    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjfd7\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.304589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.304564    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.304711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.304693    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.306618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.306595    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.325232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.325207    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:27:57.329301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.329282    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjfd7\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7\") pod \"payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") " pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.507969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.507939    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:57.819056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:57.818991    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"]
Oct 30 01:27:57.822162 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:27:57.822132    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e571fa_e95e_43ad_b4cf_44cb59f16f83.slice/crio-977204febbaf39cca6abf3db025998e88f1e4ad9c78d5f99bc4c1290ec40e450 WatchSource:0}: Error finding container 977204febbaf39cca6abf3db025998e88f1e4ad9c78d5f99bc4c1290ec40e450: Status 404 returned error can't find the container with id 977204febbaf39cca6abf3db025998e88f1e4ad9c78d5f99bc4c1290ec40e450
Oct 30 01:27:58.129600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:58.129489    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k" event={"ID":"e1e571fa-e95e-43ad-b4cf-44cb59f16f83","Type":"ContainerStarted","Data":"73cc0bd681cd3ff789f606323bf4ada38bc4b263f8e8a8969c86e9cf71b8c51d"}
Oct 30 01:27:58.129600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:58.129563    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:27:58.129600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:58.129586    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k" event={"ID":"e1e571fa-e95e-43ad-b4cf-44cb59f16f83","Type":"ContainerStarted","Data":"977204febbaf39cca6abf3db025998e88f1e4ad9c78d5f99bc4c1290ec40e450"}
Oct 30 01:27:58.175414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:58.175358    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k" podStartSLOduration=2.175344127 podStartE2EDuration="2.175344127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:27:58.174417473 +0000 UTC m=+2786.203727706" watchObservedRunningTime="2026-10-30 01:27:58.175344127 +0000 UTC m=+2786.204654358"
Oct 30 01:27:59.123347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:27:59.123285    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:00.928797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:00.928766    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:03.117142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:03.117110    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:04.924967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:04.924918    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:05.372207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:05.372150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:05.729505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:05.729466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:07.023237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:07.023201    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:08.819610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:08.819561    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:09.139735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:09.139659    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:28:10.621520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:10.621485    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:12.409935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:12.409899    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:14.232968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:14.232935    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:16.013528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:16.013487    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:17.922473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:17.922437    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:19.756718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:19.756678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:21.543965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:21.543909    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:23.510838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:23.510795    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:23.735937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:23.735898    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:25.311520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:25.311471    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:27.010785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:27.010755    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:27.016030 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:28:27.015998    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:28:28.286379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:28.286346    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"]
Oct 30 01:28:28.323150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:28.323118    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:28.479451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:28.479408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:29.103052 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:29.103019    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:30.972023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:30.971982    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:32.769151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:32.769113    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:33.287520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.287457    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k" podUID="e1e571fa-e95e-43ad-b4cf-44cb59f16f83" containerName="payload-processing" containerID="cri-o://73cc0bd681cd3ff789f606323bf4ada38bc4b263f8e8a8969c86e9cf71b8c51d" gracePeriod=25
Oct 30 01:28:33.600581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.529445    4577 generic.go:356] "Generic (PLEG): container finished" podID="e1e571fa-e95e-43ad-b4cf-44cb59f16f83" containerID="73cc0bd681cd3ff789f606323bf4ada38bc4b263f8e8a8969c86e9cf71b8c51d" exitCode=0
Oct 30 01:28:33.600581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.529510    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k" event={"ID":"e1e571fa-e95e-43ad-b4cf-44cb59f16f83","Type":"ContainerDied","Data":"73cc0bd681cd3ff789f606323bf4ada38bc4b263f8e8a8969c86e9cf71b8c51d"}
Oct 30 01:28:33.625220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.625195    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:28:33.743509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.743476    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume\") pod \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") "
Oct 30 01:28:33.743704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.743569    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7\") pod \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") "
Oct 30 01:28:33.743704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.743648    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials\") pod \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") "
Oct 30 01:28:33.743873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.743740    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume\") pod \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\" (UID: \"e1e571fa-e95e-43ad-b4cf-44cb59f16f83\") "
Oct 30 01:28:33.744084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.743952    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume" pod "e1e571fa-e95e-43ad-b4cf-44cb59f16f83" (UID: "e1e571fa-e95e-43ad-b4cf-44cb59f16f83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:33.744210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.744110    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume" pod "e1e571fa-e95e-43ad-b4cf-44cb59f16f83" (UID: "e1e571fa-e95e-43ad-b4cf-44cb59f16f83"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:33.744712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.744688    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:28:33.744809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.744718    4577 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-routing-overlay-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:28:33.746285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.746255    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials" pod "e1e571fa-e95e-43ad-b4cf-44cb59f16f83" (UID: "e1e571fa-e95e-43ad-b4cf-44cb59f16f83"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:33.746419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.746396    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7" pod "e1e571fa-e95e-43ad-b4cf-44cb59f16f83" (UID: "e1e571fa-e95e-43ad-b4cf-44cb59f16f83"). InnerVolumeSpecName "kube-api-access-pjfd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:33.846620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.846582    4577 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-provider-credentials\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:28:33.846620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:33.846621    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjfd7\" (UniqueName: \"kubernetes.io/projected/e1e571fa-e95e-43ad-b4cf-44cb59f16f83-kube-api-access-pjfd7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:28:34.539083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.539052    4577 generic.go:356] "Generic (PLEG): container finished" podID="aa6ec37d-8703-4491-a295-91eb0e031794" containerID="e0780bb0f3158782707471cc027dc31d76daf852dcf4aa79ec15fbce11d7ce2c" exitCode=0
Oct 30 01:28:34.539289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.539130    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" event={"ID":"aa6ec37d-8703-4491-a295-91eb0e031794","Type":"ContainerDied","Data":"e0780bb0f3158782707471cc027dc31d76daf852dcf4aa79ec15fbce11d7ce2c"}
Oct 30 01:28:34.539660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.539637    4577 scope.go:122] "RemoveContainer" containerID="e0780bb0f3158782707471cc027dc31d76daf852dcf4aa79ec15fbce11d7ce2c"
Oct 30 01:28:34.540745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.540725    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k" event={"ID":"e1e571fa-e95e-43ad-b4cf-44cb59f16f83","Type":"ContainerDied","Data":"977204febbaf39cca6abf3db025998e88f1e4ad9c78d5f99bc4c1290ec40e450"}
Oct 30 01:28:34.540859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.540778    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"
Oct 30 01:28:34.540932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.540783    4577 scope.go:122] "RemoveContainer" containerID="73cc0bd681cd3ff789f606323bf4ada38bc4b263f8e8a8969c86e9cf71b8c51d"
Oct 30 01:28:34.541739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.541698    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:28:34.541880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.541801    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:28:34.541990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.541889    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:28:34.612350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.612321    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:34.643969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.643753    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"]
Oct 30 01:28:34.653880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:34.649363    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-1afb3e/payload-processing-external-model-e2e-extmod-b-1afb3e-74ffk2d2k"]
Oct 30 01:28:35.579031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:35.578993    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" event={"ID":"aa6ec37d-8703-4491-a295-91eb0e031794","Type":"ContainerStarted","Data":"2c4a5ebea899c1fc6c219385d1f62560fe692da78e809b5f71e0828ce15d3b88"}
Oct 30 01:28:36.025015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:36.024984    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:28:36.025209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:36.025181    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused" start-of-body=
Oct 30 01:28:36.025261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:36.025226    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused"
Oct 30 01:28:36.112827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:36.112800    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1e571fa-e95e-43ad-b4cf-44cb59f16f83" path="/var/lib/kubelet/pods/e1e571fa-e95e-43ad-b4cf-44cb59f16f83/volumes"
Oct 30 01:28:36.372101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:36.372014    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:37.025322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:37.025274    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused" start-of-body=
Oct 30 01:28:37.025729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:37.025342    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused"
Oct 30 01:28:37.371110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:37.371001    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:37.527625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:37.527589    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:38.025110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:38.025074    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused" start-of-body=
Oct 30 01:28:38.025302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:38.025130    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused"
Oct 30 01:28:38.313576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:38.313473    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:38.987552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:38.987519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:39.025022 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:39.024982    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused" start-of-body=
Oct 30 01:28:39.025226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:39.025054    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused"
Oct 30 01:28:40.024910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:40.024836    4577 patch_prober.go:29] interesting pod/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused" start-of-body=
Oct 30 01:28:40.025330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:40.024954    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.235:15021/healthz/ready\": dial tcp 10.134.0.235:15021: connect: connection refused"
Oct 30 01:28:40.123797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:40.123765    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:40.998993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:40.998959    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:28:41.030493 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:41.030463    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:28:41.631489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:41.631409    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:28:41.909554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:41.909475    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:43.739090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:43.739003    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:44.455365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:44.455322    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:45.657038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:45.657003    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:45.823871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:45.819703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:45.972755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:45.972666    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:46.211945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:46.211908    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:46.357989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:46.357926    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:46.519092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:46.519052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:47.895741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:47.895707    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:49.702604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:49.702565    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:51.517231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:51.517194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:53.229082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:53.229053    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:53.411344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:53.411296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:53.604484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:53.604378    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:53.738163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:53.738092    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:53.922819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:53.922679    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:54.111905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:54.111871    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:54.829227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:54.829174    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:54.978246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:54.978211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:55.505431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:55.505394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:56.247426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:56.247395    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:56.376582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:56.376549    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:56.552332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:56.552246    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:57.130102 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:28:57.130059    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:28:57.567289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:57.567242    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:57.702798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:57.702763    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:28:59.415714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:28:59.415682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:01.202683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:01.202645    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:03.023162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:03.023124    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:04.834395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:04.834361    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:06.633199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:06.633164    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:08.438718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:08.438673    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:10.232009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:10.231977    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:11.581042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:11.580994    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:11.801205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:11.801170    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:11.920790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:11.920709    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:12.116776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:12.116737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:12.754639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:12.754604    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:12.865500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:12.865458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:13.072699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.072617    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"]
Oct 30 01:29:13.073410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.073393    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e1e571fa-e95e-43ad-b4cf-44cb59f16f83" containerName="payload-processing"
Oct 30 01:29:13.073475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.073411    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1e571fa-e95e-43ad-b4cf-44cb59f16f83" containerName="payload-processing"
Oct 30 01:29:13.073526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.073517    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1e571fa-e95e-43ad-b4cf-44cb59f16f83" containerName="payload-processing"
Oct 30 01:29:13.076327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.076308    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.109378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.109344    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"payload-processing-external-model-plugins-e2e-extmod-a-c7bba9\""
Oct 30 01:29:13.109936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.109784    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"openshift-service-ca.crt\""
Oct 30 01:29:13.109936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.109812    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"kube-root-ca.crt\""
Oct 30 01:29:13.109936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.109834    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"routing-overlay\""
Oct 30 01:29:13.109936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.109906    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"payload-processing-external-model-e2e--42876bb9-dockercfg-sskmf\""
Oct 30 01:29:13.109936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.109913    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"agc-cred-812a725ec060\""
Oct 30 01:29:13.113811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.113787    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"]
Oct 30 01:29:13.227143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.198447    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.227143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.198647    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mxwf\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.227143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.198685    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.227143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.198769    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.277615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.277578    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"]
Oct 30 01:29:13.282573 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.282527    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5mxwf plugins-config-volume provider-credentials routing-overlay-volume], unattached volumes=[], failed to process volumes=[]: context canceled" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb" podUID="2081e2df-ad80-4723-8417-59a3c2f6554b"
Oct 30 01:29:13.307337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.304832    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mxwf\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.307337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.304912    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.307337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.306362    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.320265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.317002    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.320265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.317292    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.320265 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.317610    4577 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-c7bba9/agc-cred-812a725ec060: secret "agc-cred-812a725ec060" not found
Oct 30 01:29:13.320265 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.317629    4577 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb: secret "agc-cred-812a725ec060" not found
Oct 30 01:29:13.320265 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.317700    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials podName:2081e2df-ad80-4723-8417-59a3c2f6554b nodeName:}" failed. No retries permitted until 2026-10-30 01:29:13.817677861 +0000 UTC m=+2861.846988087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb" (UID: "2081e2df-ad80-4723-8417-59a3c2f6554b") : secret "agc-cred-812a725ec060" not found
Oct 30 01:29:13.320265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.318640    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.335580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.333708    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:13.359261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.341468    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mxwf\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.450449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.450387    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:13.834410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.834375    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:13.834984 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.834557    4577 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-c7bba9/agc-cred-812a725ec060: secret "agc-cred-812a725ec060" not found
Oct 30 01:29:13.834984 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.834582    4577 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb: secret "agc-cred-812a725ec060" not found
Oct 30 01:29:13.834984 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:13.834673    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials podName:2081e2df-ad80-4723-8417-59a3c2f6554b nodeName:}" failed. No retries permitted until 2026-10-30 01:29:14.83464994 +0000 UTC m=+2862.863960168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb" (UID: "2081e2df-ad80-4723-8417-59a3c2f6554b") : secret "agc-cred-812a725ec060" not found
Oct 30 01:29:13.999269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:13.999235    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:14.003040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.003012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:14.007717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.007694    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:14.050564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.048183    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume\") pod \"2081e2df-ad80-4723-8417-59a3c2f6554b\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") "
Oct 30 01:29:14.050564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.048239    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf\") pod \"2081e2df-ad80-4723-8417-59a3c2f6554b\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") "
Oct 30 01:29:14.050564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.048287    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume\") pod \"2081e2df-ad80-4723-8417-59a3c2f6554b\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") "
Oct 30 01:29:14.051749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.051417    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume" pod "2081e2df-ad80-4723-8417-59a3c2f6554b" (UID: "2081e2df-ad80-4723-8417-59a3c2f6554b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:29:14.053011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.052719    4577 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-routing-overlay-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:29:14.053417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.053395    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume" pod "2081e2df-ad80-4723-8417-59a3c2f6554b" (UID: "2081e2df-ad80-4723-8417-59a3c2f6554b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:29:14.093149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.093065    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf" pod "2081e2df-ad80-4723-8417-59a3c2f6554b" (UID: "2081e2df-ad80-4723-8417-59a3c2f6554b"). InnerVolumeSpecName "kube-api-access-5mxwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:29:14.153991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.153960    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mxwf\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-kube-api-access-5mxwf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:29:14.153991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.153989    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2081e2df-ad80-4723-8417-59a3c2f6554b-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:29:14.864818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:14.864764    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb\" (UID: \"2081e2df-ad80-4723-8417-59a3c2f6554b\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:14.865383 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:14.864942    4577 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-c7bba9/agc-cred-812a725ec060: secret "agc-cred-812a725ec060" not found
Oct 30 01:29:14.865383 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:14.864963    4577 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb: secret "agc-cred-812a725ec060" not found
Oct 30 01:29:14.865383 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:14.865023    4577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials podName:2081e2df-ad80-4723-8417-59a3c2f6554b nodeName:}" failed. No retries permitted until 2026-10-30 01:29:16.86500462 +0000 UTC m=+2864.894314847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb" (UID: "2081e2df-ad80-4723-8417-59a3c2f6554b") : secret "agc-cred-812a725ec060" not found
Oct 30 01:29:15.008167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:15.008134    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"
Oct 30 01:29:15.104901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:15.104692    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"]
Oct 30 01:29:15.111204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:15.111159    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-958dt2kvb"]
Oct 30 01:29:15.171557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:15.171446    4577 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2081e2df-ad80-4723-8417-59a3c2f6554b-provider-credentials\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:29:15.834399 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:15.834367    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:16.117671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:16.117580    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2081e2df-ad80-4723-8417-59a3c2f6554b" path="/var/lib/kubelet/pods/2081e2df-ad80-4723-8417-59a3c2f6554b/volumes"
Oct 30 01:29:17.861172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:17.861138    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:18.925502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:18.925474    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:19.047147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:19.047114    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:19.559090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:19.559046    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:19.728161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:19.728131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:19.840748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:19.840652    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:19.960234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:19.960200    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:21.586396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:21.586366    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:23.418291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:23.418255    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:23.837340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:23.837297    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:25.208612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:25.208578    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:26.399464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:26.399427    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:26.529746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:26.529717    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:26.946769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:26.946733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:27.017493 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:27.017460    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:29:27.075752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:27.075578    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:27.861355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:27.861270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:27.981736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:27.981696    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:29.013137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:29.013105    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:29.191305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:29.191259    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:30.822781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:30.822754    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:32.017140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:32.017108    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:32.127094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:32.127061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:32.638489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:32.638399    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:32.748110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:32.748079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:34.528118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:34.528091    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:36.311895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:36.311822    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:36.559027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:36.558996    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:38.116618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:38.116586    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:38.938776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:38.938744    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:39.060585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:39.060533    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:39.175559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:39.175528    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:39.902586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:39.902551    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:41.607733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:41.607692    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:42.915728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:42.915692    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:43.123144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:43.123113    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:43.402638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:43.402601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:43.856951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:43.856916    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:43.973194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:43.973157    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:45.354993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:45.354958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:45.787587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:45.787554    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:45.915045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:45.915013    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:46.726307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:46.726267    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:46.859197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:46.859161    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:47.353032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:47.352986    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:49.194988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:49.194958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:51.004010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:51.003965    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:52.804976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:52.804943    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:52.944080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:52.944048    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:53.055859 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.055750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:53.322370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.322294    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"]
Oct 30 01:29:53.327049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.327027    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.346936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.346909    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"]
Oct 30 01:29:53.355165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.355141    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 01:29:53.355376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.355163    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-0e3c0a7229d2\""
Oct 30 01:29:53.355376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.355214    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-06d7d794cf59\""
Oct 30 01:29:53.359096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.355262    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 01:29:53.359096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.355277    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 01:29:53.359474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.359450    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-drckg\""
Oct 30 01:29:53.359630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.359483    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 01:29:53.478882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.478821    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.493871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.479816    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.494200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.494170    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.494420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.494401    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grwpc\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.618035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.597031    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.618035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.597097    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.618035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.597178    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grwpc\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.618035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.597284    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.618035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.615492    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.618035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.616423    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.645584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.645545    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grwpc\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.656743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.651993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials\") pod \"payload-processing-external-model-79c9fb4bbb-tdn2g\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") " pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:53.947912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:53.947821    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:54.271875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.271807    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"]
Oct 30 01:29:54.272651 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:29:54.272623    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868db612_a024_4643_bea7_1c0844476d8f.slice/crio-5791e8f3b9d1127ef216873e165a1a3ef9745bba00757a3372ff9577a7ee9e4a WatchSource:0}: Error finding container 5791e8f3b9d1127ef216873e165a1a3ef9745bba00757a3372ff9577a7ee9e4a: Status 404 returned error can't find the container with id 5791e8f3b9d1127ef216873e165a1a3ef9745bba00757a3372ff9577a7ee9e4a
Oct 30 01:29:54.664417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.664306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:54.686162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.686124    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g" event={"ID":"868db612-a024-4643-bea7-1c0844476d8f","Type":"ContainerStarted","Data":"2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919"}
Oct 30 01:29:54.686345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.686186    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g" event={"ID":"868db612-a024-4643-bea7-1c0844476d8f","Type":"ContainerStarted","Data":"5791e8f3b9d1127ef216873e165a1a3ef9745bba00757a3372ff9577a7ee9e4a"}
Oct 30 01:29:54.686345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.686218    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:29:54.730805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.730752    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g" podStartSLOduration=1.730737867 podStartE2EDuration="1.730737867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:29:54.729449718 +0000 UTC m=+2902.758759950" watchObservedRunningTime="2026-10-30 01:29:54.730737867 +0000 UTC m=+2902.760048097"
Oct 30 01:29:54.776448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.776408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:54.890828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:54.890768    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:55.636241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:55.636207    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"]
Oct 30 01:29:56.721162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:56.721131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:29:57.015806 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:29:57.015767    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:29:58.621286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:29:58.621222    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:00.177728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.177692    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"]
Oct 30 01:30:00.183333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.183307    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.216793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.214217    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"]
Oct 30 01:30:00.225099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.225072    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.234768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.234738    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"]
Oct 30 01:30:00.243387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.243359    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"]
Oct 30 01:30:00.296991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.276009    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:30:00.296991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.278470    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.296991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.284300    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"]
Oct 30 01:30:00.304612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.302388    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"]
Oct 30 01:30:00.335693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.335655    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.335918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.335731    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhkpj\" (UniqueName: \"kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.335918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.335767    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-759b9\" (UniqueName: \"kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.336056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.335969    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.339183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.339150    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"]
Oct 30 01:30:00.340181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.340157    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.351063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.350992    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"]
Oct 30 01:30:00.358075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.358052    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"]
Oct 30 01:30:00.358322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.358296    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.360691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.360671    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"]
Oct 30 01:30:00.370682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.370661    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"]
Oct 30 01:30:00.370905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.370774    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.376548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.376521    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"]
Oct 30 01:30:00.376670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.376567    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"]
Oct 30 01:30:00.376670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.376649    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.383358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.383334    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"]
Oct 30 01:30:00.437667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.437595    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888730-49dlg\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.437875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.437724    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.437875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.437789    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kccx\" (UniqueName: \"kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx\") pod \"maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.437875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.437836    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.438097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438079    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg96v\" (UniqueName: \"kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v\") pod \"maas-api-key-cleanup-29888730-49dlg\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.438166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438132    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htsps\" (UniqueName: \"kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps\") pod \"maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.438300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438273    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.438347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438335    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.438462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438390    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhkpj\" (UniqueName: \"kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.438462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438407    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-759b9\" (UniqueName: \"kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.438462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.438431    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.439068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.439046    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.463984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.463961    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhkpj\" (UniqueName: \"kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj\") pod \"maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.464138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.463962    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-759b9\" (UniqueName: \"kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.528305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.528277    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:00.535908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.535881    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:00.539552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539520    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.539709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539586    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888730-49dlg\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.539709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539673    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.539893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539784    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kccx\" (UniqueName: \"kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx\") pod \"maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.539893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539816    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.539893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539882    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zphhw\" (UniqueName: \"kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.540080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539923    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5lnb\" (UniqueName: \"kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.540080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.539957    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg96v\" (UniqueName: \"kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v\") pod \"maas-api-key-cleanup-29888730-49dlg\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.540300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.540275    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htsps\" (UniqueName: \"kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps\") pod \"maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.540386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.540368    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.540499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.540286    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888730-49dlg\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.540833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.540812    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.541214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.541192    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.583001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.582946    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:00.606877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.600968    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg96v\" (UniqueName: \"kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v\") pod \"maas-api-key-cleanup-29888730-49dlg\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.630722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.623676    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htsps\" (UniqueName: \"kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps\") pod \"maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.630722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.624448    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kccx\" (UniqueName: \"kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx\") pod \"maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.632603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.632571    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:00.642548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.642519    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.642677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.642627    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zphhw\" (UniqueName: \"kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.642677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.642662    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5lnb\" (UniqueName: \"kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.642951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.642784    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.643439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.643374    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.643839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.643818    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.699449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.675599    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5lnb\" (UniqueName: \"kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb\") pod \"maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.699449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.679096    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:00.699449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.695348    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:00.699449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.697274    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:00.702268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.702238    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zphhw\" (UniqueName: \"kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.704430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.704404    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:00.910979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.909212    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"]
Oct 30 01:30:00.912280 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:00.912248    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b976102_d3f2_4f24_80ea_bec4ff8610d6.slice/crio-1e5d800b6ca06770f9172014f2a589fb3b6f9f78b3c9dac8692bea00a7b0b078 WatchSource:0}: Error finding container 1e5d800b6ca06770f9172014f2a589fb3b6f9f78b3c9dac8692bea00a7b0b078: Status 404 returned error can't find the container with id 1e5d800b6ca06770f9172014f2a589fb3b6f9f78b3c9dac8692bea00a7b0b078
Oct 30 01:30:00.973366 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:00.973302    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a8c176_24e1_458f_ac92_6221ca47e51b.slice/crio-f75a6c8627e947cd70dce31ecce83239eacac19d879dafb2cb90edb5e90240a1 WatchSource:0}: Error finding container f75a6c8627e947cd70dce31ecce83239eacac19d879dafb2cb90edb5e90240a1: Status 404 returned error can't find the container with id f75a6c8627e947cd70dce31ecce83239eacac19d879dafb2cb90edb5e90240a1
Oct 30 01:30:00.981432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:00.981389    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"]
Oct 30 01:30:01.009539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.009512    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"]
Oct 30 01:30:01.035591 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:01.035559    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184798c0_e246_4b22_b007_e78e7318cf8c.slice/crio-8258766a5743e0afe8a7139989b6a7279c1f1c7ea2daabef915d3a9833efa13a WatchSource:0}: Error finding container 8258766a5743e0afe8a7139989b6a7279c1f1c7ea2daabef915d3a9833efa13a: Status 404 returned error can't find the container with id 8258766a5743e0afe8a7139989b6a7279c1f1c7ea2daabef915d3a9833efa13a
Oct 30 01:30:01.082532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.082373    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"]
Oct 30 01:30:01.151137 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:01.151097    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00bc966_91f0_47c4_955b_aeace632d6eb.slice/crio-6e77a86ab5903fe3ecb83df5c987c396c3664743d86dc7af8622d5da64ff388b WatchSource:0}: Error finding container 6e77a86ab5903fe3ecb83df5c987c396c3664743d86dc7af8622d5da64ff388b: Status 404 returned error can't find the container with id 6e77a86ab5903fe3ecb83df5c987c396c3664743d86dc7af8622d5da64ff388b
Oct 30 01:30:01.159308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.159268    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"]
Oct 30 01:30:01.159921 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:01.159895    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01201aae_0a83_48c8_9232_45278008a716.slice/crio-e3fd8bff37cc2dc41cede0d2191250e9ec9a40661e9a0dd45c2359db2bbd82b5 WatchSource:0}: Error finding container e3fd8bff37cc2dc41cede0d2191250e9ec9a40661e9a0dd45c2359db2bbd82b5: Status 404 returned error can't find the container with id e3fd8bff37cc2dc41cede0d2191250e9ec9a40661e9a0dd45c2359db2bbd82b5
Oct 30 01:30:01.317401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.317368    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"]
Oct 30 01:30:01.321430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.321404    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"]
Oct 30 01:30:01.363583 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:01.361345    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1230d7f0_c1f2_4674_9a06_e360ae131d59.slice/crio-5fdc55e78e986e8d0d1a1d1bcc3749db4ee78ca5fc29d8aa1fbb1ade1b0421fd WatchSource:0}: Error finding container 5fdc55e78e986e8d0d1a1d1bcc3749db4ee78ca5fc29d8aa1fbb1ade1b0421fd: Status 404 returned error can't find the container with id 5fdc55e78e986e8d0d1a1d1bcc3749db4ee78ca5fc29d8aa1fbb1ade1b0421fd
Oct 30 01:30:01.363583 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:01.363219    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a8c9e3_563c_4581_841c_c9990a08665e.slice/crio-453ace4875a6e70b05b5180c4d93f1b1bb6eeee3f08a123d82cb66f98e2f6ceb WatchSource:0}: Error finding container 453ace4875a6e70b05b5180c4d93f1b1bb6eeee3f08a123d82cb66f98e2f6ceb: Status 404 returned error can't find the container with id 453ace4875a6e70b05b5180c4d93f1b1bb6eeee3f08a123d82cb66f98e2f6ceb
Oct 30 01:30:01.708489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.708452    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g" podUID="868db612-a024-4643-bea7-1c0844476d8f" containerName="payload-processing" containerID="cri-o://2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919" gracePeriod=25
Oct 30 01:30:01.813370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.813341    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerStarted","Data":"5fdc55e78e986e8d0d1a1d1bcc3749db4ee78ca5fc29d8aa1fbb1ade1b0421fd"}
Oct 30 01:30:01.815299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.815271    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerStarted","Data":"ac8264cde9f7107fe2b5c324ba3201968e90c8864619da9a68dae53fcf6677c1"}
Oct 30 01:30:01.815414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.815323    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerStarted","Data":"8258766a5743e0afe8a7139989b6a7279c1f1c7ea2daabef915d3a9833efa13a"}
Oct 30 01:30:01.816905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.816877    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerStarted","Data":"453ace4875a6e70b05b5180c4d93f1b1bb6eeee3f08a123d82cb66f98e2f6ceb"}
Oct 30 01:30:01.818735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.818713    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerStarted","Data":"6e77a86ab5903fe3ecb83df5c987c396c3664743d86dc7af8622d5da64ff388b"}
Oct 30 01:30:01.820559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.820533    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerStarted","Data":"4f9b0a76302743725db194c01ea9667cf706db95555453176d89daea5cca2a75"}
Oct 30 01:30:01.820646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.820582    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerStarted","Data":"1e5d800b6ca06770f9172014f2a589fb3b6f9f78b3c9dac8692bea00a7b0b078"}
Oct 30 01:30:01.822680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.822094    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerStarted","Data":"22307242f4f594880334cd7845757e6f2da4c2ad50bbc6965299affb95d6bafa"}
Oct 30 01:30:01.822680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.822140    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerStarted","Data":"e3fd8bff37cc2dc41cede0d2191250e9ec9a40661e9a0dd45c2359db2bbd82b5"}
Oct 30 01:30:01.823732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.823712    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerStarted","Data":"9ac254dc7b69c0424c967b6fa75853374760f51a894418004f57d70fa7f057b4"}
Oct 30 01:30:01.823829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.823748    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerStarted","Data":"f75a6c8627e947cd70dce31ecce83239eacac19d879dafb2cb90edb5e90240a1"}
Oct 30 01:30:01.866898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.866802    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" podStartSLOduration=1.866782203 podStartE2EDuration="1.866782203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:01.859696452 +0000 UTC m=+2909.889006684" watchObservedRunningTime="2026-10-30 01:30:01.866782203 +0000 UTC m=+2909.896092435"
Oct 30 01:30:01.906373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.906307    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" podStartSLOduration=1.906292667 podStartE2EDuration="1.906292667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:01.902356184 +0000 UTC m=+2909.931666415" watchObservedRunningTime="2026-10-30 01:30:01.906292667 +0000 UTC m=+2909.935602899"
Oct 30 01:30:01.963891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.954194    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" podStartSLOduration=1.9541744680000002 podStartE2EDuration="1.954174468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:01.946664173 +0000 UTC m=+2909.975974405" watchObservedRunningTime="2026-10-30 01:30:01.954174468 +0000 UTC m=+2909.983484720"
Oct 30 01:30:01.993260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:01.993203    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" podStartSLOduration=1.993177086 podStartE2EDuration="1.993177086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:01.989551275 +0000 UTC m=+2910.018861507" watchObservedRunningTime="2026-10-30 01:30:01.993177086 +0000 UTC m=+2910.022487318"
Oct 30 01:30:02.083433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.083404    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:30:02.176576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.176529    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume\") pod \"868db612-a024-4643-bea7-1c0844476d8f\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") "
Oct 30 01:30:02.176765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.176715    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials\") pod \"868db612-a024-4643-bea7-1c0844476d8f\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") "
Oct 30 01:30:02.176915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.176890    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume\") pod \"868db612-a024-4643-bea7-1c0844476d8f\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") "
Oct 30 01:30:02.177080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.177059    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc\") pod \"868db612-a024-4643-bea7-1c0844476d8f\" (UID: \"868db612-a024-4643-bea7-1c0844476d8f\") "
Oct 30 01:30:02.182930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.181866    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume" pod "868db612-a024-4643-bea7-1c0844476d8f" (UID: "868db612-a024-4643-bea7-1c0844476d8f"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:02.182930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.179153    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume" pod "868db612-a024-4643-bea7-1c0844476d8f" (UID: "868db612-a024-4643-bea7-1c0844476d8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:02.251654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.251597    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc" pod "868db612-a024-4643-bea7-1c0844476d8f" (UID: "868db612-a024-4643-bea7-1c0844476d8f"). InnerVolumeSpecName "kube-api-access-grwpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:02.257435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.257370    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials" pod "868db612-a024-4643-bea7-1c0844476d8f" (UID: "868db612-a024-4643-bea7-1c0844476d8f"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:02.280785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.280744    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grwpc\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-kube-api-access-grwpc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:02.280785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.280777    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:02.280785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.280789    4577 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/868db612-a024-4643-bea7-1c0844476d8f-provider-credentials\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:02.281096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.280798    4577 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/868db612-a024-4643-bea7-1c0844476d8f-routing-overlay-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:02.308740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.308714    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:02.835293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.835259    4577 generic.go:356] "Generic (PLEG): container finished" podID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerID="92eeca6c393062818c8a8915ddf3054f8858e4ad593409a3e72afff9ae5ed0e1" exitCode=60
Oct 30 01:30:02.835768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.835352    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerDied","Data":"92eeca6c393062818c8a8915ddf3054f8858e4ad593409a3e72afff9ae5ed0e1"}
Oct 30 01:30:02.835768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.835664    4577 scope.go:122] "RemoveContainer" containerID="92eeca6c393062818c8a8915ddf3054f8858e4ad593409a3e72afff9ae5ed0e1"
Oct 30 01:30:02.837355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.837325    4577 generic.go:356] "Generic (PLEG): container finished" podID="184798c0-e246-4b22-b007-e78e7318cf8c" containerID="ac8264cde9f7107fe2b5c324ba3201968e90c8864619da9a68dae53fcf6677c1" exitCode=60
Oct 30 01:30:02.837480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.837408    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerDied","Data":"ac8264cde9f7107fe2b5c324ba3201968e90c8864619da9a68dae53fcf6677c1"}
Oct 30 01:30:02.837745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.837652    4577 scope.go:122] "RemoveContainer" containerID="ac8264cde9f7107fe2b5c324ba3201968e90c8864619da9a68dae53fcf6677c1"
Oct 30 01:30:02.839639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.839610    4577 generic.go:356] "Generic (PLEG): container finished" podID="868db612-a024-4643-bea7-1c0844476d8f" containerID="2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919" exitCode=0
Oct 30 01:30:02.839733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.839718    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g" event={"ID":"868db612-a024-4643-bea7-1c0844476d8f","Type":"ContainerDied","Data":"2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919"}
Oct 30 01:30:02.839772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.839758    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g" event={"ID":"868db612-a024-4643-bea7-1c0844476d8f","Type":"ContainerDied","Data":"5791e8f3b9d1127ef216873e165a1a3ef9745bba00757a3372ff9577a7ee9e4a"}
Oct 30 01:30:02.839930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.839784    4577 scope.go:122] "RemoveContainer" containerID="2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919"
Oct 30 01:30:02.839971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.839945    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"
Oct 30 01:30:02.842789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.842765    4577 generic.go:356] "Generic (PLEG): container finished" podID="65a8c9e3-563c-4581-841c-c9990a08665e" containerID="303d209249df641cba36c9ff170fd44bbe069b312185df37377b36fca62f1972" exitCode=60
Oct 30 01:30:02.842923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.842893    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerDied","Data":"303d209249df641cba36c9ff170fd44bbe069b312185df37377b36fca62f1972"}
Oct 30 01:30:02.843253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.843233    4577 scope.go:122] "RemoveContainer" containerID="303d209249df641cba36c9ff170fd44bbe069b312185df37377b36fca62f1972"
Oct 30 01:30:02.846077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.845481    4577 generic.go:356] "Generic (PLEG): container finished" podID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerID="3b08a47ac896808a3c9b5368a25bb282056982664b0579696e20317f7eb797be" exitCode=60
Oct 30 01:30:02.846077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.845614    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerDied","Data":"3b08a47ac896808a3c9b5368a25bb282056982664b0579696e20317f7eb797be"}
Oct 30 01:30:02.846077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.845980    4577 scope.go:122] "RemoveContainer" containerID="3b08a47ac896808a3c9b5368a25bb282056982664b0579696e20317f7eb797be"
Oct 30 01:30:02.848235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.848215    4577 generic.go:356] "Generic (PLEG): container finished" podID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerID="4f9b0a76302743725db194c01ea9667cf706db95555453176d89daea5cca2a75" exitCode=60
Oct 30 01:30:02.848339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.848298    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerDied","Data":"4f9b0a76302743725db194c01ea9667cf706db95555453176d89daea5cca2a75"}
Oct 30 01:30:02.848532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.848518    4577 scope.go:122] "RemoveContainer" containerID="4f9b0a76302743725db194c01ea9667cf706db95555453176d89daea5cca2a75"
Oct 30 01:30:02.851938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.851918    4577 generic.go:356] "Generic (PLEG): container finished" podID="01201aae-0a83-48c8-9232-45278008a716" containerID="22307242f4f594880334cd7845757e6f2da4c2ad50bbc6965299affb95d6bafa" exitCode=60
Oct 30 01:30:02.852795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.852198    4577 scope.go:122] "RemoveContainer" containerID="22307242f4f594880334cd7845757e6f2da4c2ad50bbc6965299affb95d6bafa"
Oct 30 01:30:02.853086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.852214    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerDied","Data":"22307242f4f594880334cd7845757e6f2da4c2ad50bbc6965299affb95d6bafa"}
Oct 30 01:30:02.857188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.857168    4577 generic.go:356] "Generic (PLEG): container finished" podID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerID="9ac254dc7b69c0424c967b6fa75853374760f51a894418004f57d70fa7f057b4" exitCode=60
Oct 30 01:30:02.857326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.857251    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerDied","Data":"9ac254dc7b69c0424c967b6fa75853374760f51a894418004f57d70fa7f057b4"}
Oct 30 01:30:02.857451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.857434    4577 scope.go:122] "RemoveContainer" containerID="9ac254dc7b69c0424c967b6fa75853374760f51a894418004f57d70fa7f057b4"
Oct 30 01:30:02.859413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.859396    4577 scope.go:122] "RemoveContainer" containerID="2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919"
Oct 30 01:30:02.860729 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:02.860650    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919\": container with ID starting with 2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919 not found: ID does not exist" containerID="2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919"
Oct 30 01:30:02.860729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.860682    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919"} err="failed to get container status \"2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919\": rpc error: code = NotFound desc = could not find container \"2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919\": container with ID starting with 2b8321109f49412d4e4f62c68efa198ec0c372afa5f8bc4cbcf26c8e1586f919 not found: ID does not exist"
Oct 30 01:30:02.978715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.978685    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"]
Oct 30 01:30:02.986248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:02.986221    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-79c9fb4bbb-tdn2g"]
Oct 30 01:30:03.867733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.867702    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerStarted","Data":"886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752"}
Oct 30 01:30:03.869881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.869835    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerStarted","Data":"893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287"}
Oct 30 01:30:03.871691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.871667    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerStarted","Data":"275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e"}
Oct 30 01:30:03.873486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.873457    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerStarted","Data":"5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49"}
Oct 30 01:30:03.876390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.876366    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerStarted","Data":"e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47"}
Oct 30 01:30:03.878483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.878459    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerStarted","Data":"ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c"}
Oct 30 01:30:03.880233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.880211    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerStarted","Data":"4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267"}
Oct 30 01:30:03.997529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:03.997471    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" podStartSLOduration=3.997455256 podStartE2EDuration="3.997455256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:03.996078818 +0000 UTC m=+2912.025389062" watchObservedRunningTime="2026-10-30 01:30:03.997455256 +0000 UTC m=+2912.026765487"
Oct 30 01:30:04.042532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.042469    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" podStartSLOduration=4.042450233 podStartE2EDuration="4.042450233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:04.038653622 +0000 UTC m=+2912.067963853" watchObservedRunningTime="2026-10-30 01:30:04.042450233 +0000 UTC m=+2912.071760465"
Oct 30 01:30:04.086202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.086138    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" podStartSLOduration=4.086118904 podStartE2EDuration="4.086118904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:04.081144609 +0000 UTC m=+2912.110454840" watchObservedRunningTime="2026-10-30 01:30:04.086118904 +0000 UTC m=+2912.115429135"
Oct 30 01:30:04.126582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.126502    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="868db612-a024-4643-bea7-1c0844476d8f" path="/var/lib/kubelet/pods/868db612-a024-4643-bea7-1c0844476d8f/volumes"
Oct 30 01:30:04.139898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.139837    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:04.747950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.747902    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"]
Oct 30 01:30:04.748778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.748756    4577 cpu_manager.go:414] "containerMap: removing container" podUID="868db612-a024-4643-bea7-1c0844476d8f" containerName="payload-processing"
Oct 30 01:30:04.748778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.748781    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="868db612-a024-4643-bea7-1c0844476d8f" containerName="payload-processing"
Oct 30 01:30:04.749031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.749017    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="868db612-a024-4643-bea7-1c0844476d8f" containerName="payload-processing"
Oct 30 01:30:04.752814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.752795    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.764903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.764873    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"]
Oct 30 01:30:04.770318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.770292    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.773944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.772138    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-spn7j\""
Oct 30 01:30:04.779007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.778980    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"]
Oct 30 01:30:04.794009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.793986    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"]
Oct 30 01:30:04.830773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.830739    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.830985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.830788    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhd2q\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.830976    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831021    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831107    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831152    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831209    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831244    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831285    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831319    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831447    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831475    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831510    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tfqj\" (UniqueName: \"kubernetes.io/projected/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-kube-api-access-7tfqj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831536    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831606    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.831714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831639    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831686    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.831886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.831713    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.891211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.891177    4577 generic.go:356] "Generic (PLEG): container finished" podID="65a8c9e3-563c-4581-841c-c9990a08665e" containerID="e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47" exitCode=60
Oct 30 01:30:04.891694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.891269    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerDied","Data":"e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47"}
Oct 30 01:30:04.891694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.891332    4577 scope.go:122] "RemoveContainer" containerID="303d209249df641cba36c9ff170fd44bbe069b312185df37377b36fca62f1972"
Oct 30 01:30:04.891694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.891631    4577 scope.go:122] "RemoveContainer" containerID="e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47"
Oct 30 01:30:04.892132 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.892109    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw_odh-ai-gateway-infra(65a8c9e3-563c-4581-841c-c9990a08665e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" podUID="65a8c9e3-563c-4581-841c-c9990a08665e"
Oct 30 01:30:04.893667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.893637    4577 generic.go:356] "Generic (PLEG): container finished" podID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerID="ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c" exitCode=60
Oct 30 01:30:04.893780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.893683    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerDied","Data":"ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c"}
Oct 30 01:30:04.895504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.895476    4577 scope.go:122] "RemoveContainer" containerID="ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c"
Oct 30 01:30:04.895926 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.895900    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888730-49dlg_odh-ai-gateway-infra(f00bc966-91f0-47c4-955b-aeace632d6eb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb"
Oct 30 01:30:04.896064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.896040    4577 generic.go:356] "Generic (PLEG): container finished" podID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerID="4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267" exitCode=60
Oct 30 01:30:04.896196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.896115    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerDied","Data":"4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267"}
Oct 30 01:30:04.896414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.896399    4577 scope.go:122] "RemoveContainer" containerID="4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267"
Oct 30 01:30:04.896766 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.896734    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf_odh-ai-gateway-infra(5b976102-d3f2-4f24-80ea-bec4ff8610d6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6"
Oct 30 01:30:04.898156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.898135    4577 generic.go:356] "Generic (PLEG): container finished" podID="01201aae-0a83-48c8-9232-45278008a716" containerID="886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752" exitCode=60
Oct 30 01:30:04.898260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.898166    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerDied","Data":"886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752"}
Oct 30 01:30:04.898479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.898465    4577 scope.go:122] "RemoveContainer" containerID="886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752"
Oct 30 01:30:04.898805 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.898788    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w_odh-ai-gateway-infra(01201aae-0a83-48c8-9232-45278008a716)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" podUID="01201aae-0a83-48c8-9232-45278008a716"
Oct 30 01:30:04.900145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.900120    4577 generic.go:356] "Generic (PLEG): container finished" podID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerID="893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287" exitCode=60
Oct 30 01:30:04.900281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.900263    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerDied","Data":"893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287"}
Oct 30 01:30:04.900425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.900406    4577 scope.go:122] "RemoveContainer" containerID="893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287"
Oct 30 01:30:04.900704 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.900686    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf_odh-ai-gateway-infra(25a8c176-24e1-458f-ac92-6221ca47e51b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b"
Oct 30 01:30:04.902291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.902272    4577 generic.go:356] "Generic (PLEG): container finished" podID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerID="275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e" exitCode=60
Oct 30 01:30:04.902386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.902365    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerDied","Data":"275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e"}
Oct 30 01:30:04.902615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.902597    4577 scope.go:122] "RemoveContainer" containerID="275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e"
Oct 30 01:30:04.902940 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.902918    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4_odh-ai-gateway-infra(1230d7f0-c1f2-4674-9a06-e360ae131d59)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59"
Oct 30 01:30:04.904247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.904225    4577 generic.go:356] "Generic (PLEG): container finished" podID="184798c0-e246-4b22-b007-e78e7318cf8c" containerID="5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49" exitCode=60
Oct 30 01:30:04.904342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.904248    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerDied","Data":"5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49"}
Oct 30 01:30:04.904491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.904477    4577 scope.go:122] "RemoveContainer" containerID="5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49"
Oct 30 01:30:04.904694 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:04.904679    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98_odh-ai-gateway-infra(184798c0-e246-4b22-b007-e78e7318cf8c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" podUID="184798c0-e246-4b22-b007-e78e7318cf8c"
Oct 30 01:30:04.907419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.907396    4577 scope.go:122] "RemoveContainer" containerID="3b08a47ac896808a3c9b5368a25bb282056982664b0579696e20317f7eb797be"
Oct 30 01:30:04.924991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.924969    4577 scope.go:122] "RemoveContainer" containerID="4f9b0a76302743725db194c01ea9667cf706db95555453176d89daea5cca2a75"
Oct 30 01:30:04.933285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933259    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.933392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933320    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.933467 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933442    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.933624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933604    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.933624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933630    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.933782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933656    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhd2q\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.933782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933686    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.933782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933745    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.933782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933772    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933813    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933828    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933984    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.933995    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934065    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934120    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934177    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934225    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934333    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934529    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934587    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934623    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tfqj\" (UniqueName: \"kubernetes.io/projected/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-kube-api-access-7tfqj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.934666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934655    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.934875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.934754    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.935234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.935208    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.935914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.935753    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.936038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.936018    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.936123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.936055    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.937291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.937234    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.937761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.937735    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.937761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.937749    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.937944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.937785    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.938794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.938775    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.943741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.943720    4577 scope.go:122] "RemoveContainer" containerID="22307242f4f594880334cd7845757e6f2da4c2ad50bbc6965299affb95d6bafa"
Oct 30 01:30:04.959508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.959475    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhd2q\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.959664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.959504    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.959756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.959733    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tfqj\" (UniqueName: \"kubernetes.io/projected/d07ac1e5-f1b2-4018-9cfb-1f71f672c381-kube-api-access-7tfqj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-4x4jk\" (UID: \"d07ac1e5-f1b2-4018-9cfb-1f71f672c381\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:04.960436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.960409    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-bdcd2\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:04.990283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:04.990258    4577 scope.go:122] "RemoveContainer" containerID="9ac254dc7b69c0424c967b6fa75853374760f51a894418004f57d70fa7f057b4"
Oct 30 01:30:05.005051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.005022    4577 scope.go:122] "RemoveContainer" containerID="92eeca6c393062818c8a8915ddf3054f8858e4ad593409a3e72afff9ae5ed0e1"
Oct 30 01:30:05.019397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.019375    4577 scope.go:122] "RemoveContainer" containerID="ac8264cde9f7107fe2b5c324ba3201968e90c8864619da9a68dae53fcf6677c1"
Oct 30 01:30:05.078390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.078355    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:05.095777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.095597    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:05.419055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.418972    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"]
Oct 30 01:30:05.424289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.424258    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"]
Oct 30 01:30:05.425258 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:05.425224    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f25f85b_9c1a_4783_ba88_796c04f572ee.slice/crio-971c41ec97e3eeec7915f9d2e561ffab8e5d0a20f72036eb9d72579b3d96569e WatchSource:0}: Error finding container 971c41ec97e3eeec7915f9d2e561ffab8e5d0a20f72036eb9d72579b3d96569e: Status 404 returned error can't find the container with id 971c41ec97e3eeec7915f9d2e561ffab8e5d0a20f72036eb9d72579b3d96569e
Oct 30 01:30:05.425800 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:05.425737    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07ac1e5_f1b2_4018_9cfb_1f71f672c381.slice/crio-064d6b61696d8005ed2452755cdf4e23ac27b714e80e53508b0c01dc0cb6d4eb WatchSource:0}: Error finding container 064d6b61696d8005ed2452755cdf4e23ac27b714e80e53508b0c01dc0cb6d4eb: Status 404 returned error can't find the container with id 064d6b61696d8005ed2452755cdf4e23ac27b714e80e53508b0c01dc0cb6d4eb
Oct 30 01:30:05.428264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.428233    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:05.428351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.428299    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:05.428351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.428327    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:05.428571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.428544    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:05.428627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.428606    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:05.428732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.428644    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:05.919820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.919790    4577 scope.go:122] "RemoveContainer" containerID="5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49"
Oct 30 01:30:05.920328 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.920240    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98_odh-ai-gateway-infra(184798c0-e246-4b22-b007-e78e7318cf8c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" podUID="184798c0-e246-4b22-b007-e78e7318cf8c"
Oct 30 01:30:05.922102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.922073    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" event={"ID":"d07ac1e5-f1b2-4018-9cfb-1f71f672c381","Type":"ContainerStarted","Data":"bbff458b525ba346739554846001ec40b04d0134bc86aaed4187967616fada6e"}
Oct 30 01:30:05.922221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.922131    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" event={"ID":"d07ac1e5-f1b2-4018-9cfb-1f71f672c381","Type":"ContainerStarted","Data":"064d6b61696d8005ed2452755cdf4e23ac27b714e80e53508b0c01dc0cb6d4eb"}
Oct 30 01:30:05.924922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.924900    4577 scope.go:122] "RemoveContainer" containerID="e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47"
Oct 30 01:30:05.925278 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.925251    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw_odh-ai-gateway-infra(65a8c9e3-563c-4581-841c-c9990a08665e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" podUID="65a8c9e3-563c-4581-841c-c9990a08665e"
Oct 30 01:30:05.926934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.926909    4577 scope.go:122] "RemoveContainer" containerID="ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c"
Oct 30 01:30:05.927279 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.927251    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888730-49dlg_odh-ai-gateway-infra(f00bc966-91f0-47c4-955b-aeace632d6eb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb"
Oct 30 01:30:05.928607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.928580    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" event={"ID":"3f25f85b-9c1a-4783-ba88-796c04f572ee","Type":"ContainerStarted","Data":"97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8"}
Oct 30 01:30:05.928924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.928631    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" event={"ID":"3f25f85b-9c1a-4783-ba88-796c04f572ee","Type":"ContainerStarted","Data":"971c41ec97e3eeec7915f9d2e561ffab8e5d0a20f72036eb9d72579b3d96569e"}
Oct 30 01:30:05.931321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.931303    4577 scope.go:122] "RemoveContainer" containerID="4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267"
Oct 30 01:30:05.931658 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.931636    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf_odh-ai-gateway-infra(5b976102-d3f2-4f24-80ea-bec4ff8610d6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6"
Oct 30 01:30:05.933624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.933600    4577 scope.go:122] "RemoveContainer" containerID="886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752"
Oct 30 01:30:05.934190 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.933970    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w_odh-ai-gateway-infra(01201aae-0a83-48c8-9232-45278008a716)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" podUID="01201aae-0a83-48c8-9232-45278008a716"
Oct 30 01:30:05.935595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.935576    4577 scope.go:122] "RemoveContainer" containerID="893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287"
Oct 30 01:30:05.936020 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.935997    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf_odh-ai-gateway-infra(25a8c176-24e1-458f-ac92-6221ca47e51b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b"
Oct 30 01:30:05.937557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:05.937540    4577 scope.go:122] "RemoveContainer" containerID="275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e"
Oct 30 01:30:05.937811 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:05.937790    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4_odh-ai-gateway-infra(1230d7f0-c1f2-4674-9a06-e360ae131d59)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59"
Oct 30 01:30:06.055956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.055919    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:06.079001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.078968    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:06.080826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.080791    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:06.081033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.080949    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:06.119711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.119091    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podStartSLOduration=2.119068181 podStartE2EDuration="2.119068181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:06.109545377 +0000 UTC m=+2914.138855609" watchObservedRunningTime="2026-10-30 01:30:06.119068181 +0000 UTC m=+2914.148378413"
Oct 30 01:30:06.122224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.122123    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:06.125745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.125705    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:06.125945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.125762    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:06.369663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:06.369580    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podStartSLOduration=2.369557226 podStartE2EDuration="2.369557226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:06.344194611 +0000 UTC m=+2914.373504843" watchObservedRunningTime="2026-10-30 01:30:06.369557226 +0000 UTC m=+2914.398867462"
Oct 30 01:30:07.092322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:07.092279    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:07.092982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:07.092946    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:07.097261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:07.097231    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:07.097503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:07.097478    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:07.906942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:07.906801    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:08.091821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:08.091085    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:08.091821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:08.091146    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:08.097293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:08.097188    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:08.097293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:08.097239    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:09.080871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:09.080303    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:09.080871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:09.080372    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:09.105935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:09.105896    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:09.106465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:09.105958    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:09.781882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:09.779160    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:10.097935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:10.095960    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:10.097935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:10.096025    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:10.098193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:10.097951    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:10.098193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:10.098001    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:11.096909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.096786    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:11.097958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.097519    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:11.098239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.098221    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:11.098395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.098374    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:11.104534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.104472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:11.715581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.715542    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:11.825700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:11.825663    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:12.083355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:12.083310    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:12.083646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:12.083612    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:12.099337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:12.099298    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:12.099971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:12.099939    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:12.486316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:12.486286    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:12.601434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:12.601349    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:13.079571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:13.079485    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:13.079787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:13.079584    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:13.103289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:13.103137    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:13.103289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:13.103219    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:13.637251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:13.637215    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:14.079402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:14.079355    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-bdcd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused" start-of-body=
Oct 30 01:30:14.079608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:14.079428    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.246:15021/healthz/ready\": dial tcp 10.134.0.246:15021: connect: connection refused"
Oct 30 01:30:14.096954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:14.096912    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused" start-of-body=
Oct 30 01:30:14.097142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:14.096987    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": dial tcp 10.134.0.247:15021: connect: connection refused"
Oct 30 01:30:15.221183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.221120    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:15.222739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.222708    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:15.223213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.223191    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:30:15.304403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.304341    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:15.304943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.304913    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:15.305986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.305965    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk"
Oct 30 01:30:15.433089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:15.433046    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:16.104452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:16.104419    4577 scope.go:122] "RemoveContainer" containerID="ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c"
Oct 30 01:30:16.104688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:16.104666    4577 scope.go:122] "RemoveContainer" containerID="893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287"
Oct 30 01:30:16.104984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:16.104965    4577 scope.go:122] "RemoveContainer" containerID="4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267"
Oct 30 01:30:16.967317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:16.967286    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:17.110870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.109904    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:17.110870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.110711    4577 scope.go:122] "RemoveContainer" containerID="5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49"
Oct 30 01:30:17.131882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.128227    4577 scope.go:122] "RemoveContainer" containerID="275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e"
Oct 30 01:30:17.131882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.130079    4577 scope.go:122] "RemoveContainer" containerID="e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47"
Oct 30 01:30:17.220460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.220433    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerStarted","Data":"b0edb222ac5de74d12a53f6e820c61cd17c511faf41174bb74ebdfa8dfc31a3f"}
Oct 30 01:30:17.223900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.223868    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerStarted","Data":"31a04a9346c2675fc679f058f53094fa400af87b11c0fb69074d131c118f78ae"}
Oct 30 01:30:17.232120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.228057    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerStarted","Data":"8304312b22f576c88348e10114980a83b8d753e96a0ecc0dd0ac3d431bfda917"}
Oct 30 01:30:17.413863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:17.413803    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:18.098747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.098577    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:18.109663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.109565    4577 scope.go:122] "RemoveContainer" containerID="886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752"
Oct 30 01:30:18.212676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.212629    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:18.239534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.239496    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerStarted","Data":"96380703dbf87a1b179052d642d62e9a9ef5d11fd1acb34d0520e7b6eb8b4a3e"}
Oct 30 01:30:18.241794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.241766    4577 generic.go:356] "Generic (PLEG): container finished" podID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerID="8304312b22f576c88348e10114980a83b8d753e96a0ecc0dd0ac3d431bfda917" exitCode=60
Oct 30 01:30:18.241963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.241835    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerDied","Data":"8304312b22f576c88348e10114980a83b8d753e96a0ecc0dd0ac3d431bfda917"}
Oct 30 01:30:18.241963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.241932    4577 scope.go:122] "RemoveContainer" containerID="ab3e577f40fd240dc8178e50622cd458d00d4e366a086b23b28c91d1a5697d9c"
Oct 30 01:30:18.242288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.242272    4577 scope.go:122] "RemoveContainer" containerID="8304312b22f576c88348e10114980a83b8d753e96a0ecc0dd0ac3d431bfda917"
Oct 30 01:30:18.242640 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:18.242622    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29888730-49dlg_odh-ai-gateway-infra(f00bc966-91f0-47c4-955b-aeace632d6eb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb"
Oct 30 01:30:18.243989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.243970    4577 generic.go:356] "Generic (PLEG): container finished" podID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerID="b0edb222ac5de74d12a53f6e820c61cd17c511faf41174bb74ebdfa8dfc31a3f" exitCode=60
Oct 30 01:30:18.244097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.244035    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerDied","Data":"b0edb222ac5de74d12a53f6e820c61cd17c511faf41174bb74ebdfa8dfc31a3f"}
Oct 30 01:30:18.244430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.244413    4577 scope.go:122] "RemoveContainer" containerID="b0edb222ac5de74d12a53f6e820c61cd17c511faf41174bb74ebdfa8dfc31a3f"
Oct 30 01:30:18.244764 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:18.244744    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf_odh-ai-gateway-infra(5b976102-d3f2-4f24-80ea-bec4ff8610d6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6"
Oct 30 01:30:18.246211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.246189    4577 generic.go:356] "Generic (PLEG): container finished" podID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerID="31a04a9346c2675fc679f058f53094fa400af87b11c0fb69074d131c118f78ae" exitCode=60
Oct 30 01:30:18.246344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.246328    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerDied","Data":"31a04a9346c2675fc679f058f53094fa400af87b11c0fb69074d131c118f78ae"}
Oct 30 01:30:18.246702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.246680    4577 scope.go:122] "RemoveContainer" containerID="31a04a9346c2675fc679f058f53094fa400af87b11c0fb69074d131c118f78ae"
Oct 30 01:30:18.247050 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:18.247027    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf_odh-ai-gateway-infra(25a8c176-24e1-458f-ac92-6221ca47e51b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b"
Oct 30 01:30:18.249543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.249517    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerStarted","Data":"dc5a1b76bd3b4a3dca91dd1bd05eb4454d9c750b46f407a595ce791355e1b02d"}
Oct 30 01:30:18.252341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.252319    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerStarted","Data":"133cad569ac44fce62d9e0a3a3b37fd3299d6e34f538af3e69262b768acfb5a7"}
Oct 30 01:30:18.260972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.260935    4577 scope.go:122] "RemoveContainer" containerID="4327ccf8cedc034b1fcea43e6930fecb2e7f534b45fd98d407f4ae8361833267"
Oct 30 01:30:18.280188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.279328    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"]
Oct 30 01:30:18.293972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.293937    4577 scope.go:122] "RemoveContainer" containerID="893c245c435f98c97fa3cc09f973e52f12cca2b141f2e98d797ce3f73b3cf287"
Oct 30 01:30:18.326187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.326118    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"]
Oct 30 01:30:18.376860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.376812    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"]
Oct 30 01:30:18.933940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:18.933790    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:19.118884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.115149    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:19.266434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.266393    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerStarted","Data":"4ef0769f14f67bd11fa83a1947a77adba573561998b962a637c48a7b1391a1f6"}
Oct 30 01:30:19.271031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.271003    4577 generic.go:356] "Generic (PLEG): container finished" podID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerID="dc5a1b76bd3b4a3dca91dd1bd05eb4454d9c750b46f407a595ce791355e1b02d" exitCode=60
Oct 30 01:30:19.271145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.271125    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerDied","Data":"dc5a1b76bd3b4a3dca91dd1bd05eb4454d9c750b46f407a595ce791355e1b02d"}
Oct 30 01:30:19.271189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.271175    4577 scope.go:122] "RemoveContainer" containerID="275a7790f853d8c92f902a555db67559112f0ec01cfeb66ba37d0c23569d9f5e"
Oct 30 01:30:19.271626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.271610    4577 scope.go:122] "RemoveContainer" containerID="dc5a1b76bd3b4a3dca91dd1bd05eb4454d9c750b46f407a595ce791355e1b02d"
Oct 30 01:30:19.272092 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:19.272036    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4_odh-ai-gateway-infra(1230d7f0-c1f2-4674-9a06-e360ae131d59)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59"
Oct 30 01:30:19.275327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.275304    4577 generic.go:356] "Generic (PLEG): container finished" podID="184798c0-e246-4b22-b007-e78e7318cf8c" containerID="133cad569ac44fce62d9e0a3a3b37fd3299d6e34f538af3e69262b768acfb5a7" exitCode=60
Oct 30 01:30:19.275439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.275372    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerDied","Data":"133cad569ac44fce62d9e0a3a3b37fd3299d6e34f538af3e69262b768acfb5a7"}
Oct 30 01:30:19.275729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.275713    4577 scope.go:122] "RemoveContainer" containerID="133cad569ac44fce62d9e0a3a3b37fd3299d6e34f538af3e69262b768acfb5a7"
Oct 30 01:30:19.276060 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:19.276040    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98_odh-ai-gateway-infra(184798c0-e246-4b22-b007-e78e7318cf8c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" podUID="184798c0-e246-4b22-b007-e78e7318cf8c"
Oct 30 01:30:19.278171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.278120    4577 generic.go:356] "Generic (PLEG): container finished" podID="65a8c9e3-563c-4581-841c-c9990a08665e" containerID="96380703dbf87a1b179052d642d62e9a9ef5d11fd1acb34d0520e7b6eb8b4a3e" exitCode=60
Oct 30 01:30:19.278311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.278226    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerDied","Data":"96380703dbf87a1b179052d642d62e9a9ef5d11fd1acb34d0520e7b6eb8b4a3e"}
Oct 30 01:30:19.278572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.278551    4577 scope.go:122] "RemoveContainer" containerID="96380703dbf87a1b179052d642d62e9a9ef5d11fd1acb34d0520e7b6eb8b4a3e"
Oct 30 01:30:19.278908 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:19.278880    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw_odh-ai-gateway-infra(65a8c9e3-563c-4581-841c-c9990a08665e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" podUID="65a8c9e3-563c-4581-841c-c9990a08665e"
Oct 30 01:30:19.290140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.290066    4577 scope.go:122] "RemoveContainer" containerID="5af623b9710ba927d014c586496d797a96fbc9149fb38cb410ab5cca3a58ce49"
Oct 30 01:30:19.306046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.305998    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"]
Oct 30 01:30:19.315807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.315119    4577 scope.go:122] "RemoveContainer" containerID="e8a23334fa065a3ab8836297451269a7d82ffaf8a6093ee933249bd94c9f1b47"
Oct 30 01:30:19.401617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.401579    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"]
Oct 30 01:30:19.467203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.465816    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:19.536867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.536154    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:19.651876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.649070    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca\") pod \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") "
Oct 30 01:30:19.687869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.669395    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca" pod "5b976102-d3f2-4f24-80ea-bec4ff8610d6" (UID: "5b976102-d3f2-4f24-80ea-bec4ff8610d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:19.687869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.669688    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj\" (UniqueName: \"kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj\") pod \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\" (UID: \"5b976102-d3f2-4f24-80ea-bec4ff8610d6\") "
Oct 30 01:30:19.764869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.727484    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b976102-d3f2-4f24-80ea-bec4ff8610d6-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:19.770115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.769672    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:19.852565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.828322    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9\" (UniqueName: \"kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9\") pod \"25a8c176-24e1-458f-ac92-6221ca47e51b\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") "
Oct 30 01:30:19.852565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.828622    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca\") pod \"25a8c176-24e1-458f-ac92-6221ca47e51b\" (UID: \"25a8c176-24e1-458f-ac92-6221ca47e51b\") "
Oct 30 01:30:19.852565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.849200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9" pod "25a8c176-24e1-458f-ac92-6221ca47e51b" (UID: "25a8c176-24e1-458f-ac92-6221ca47e51b"). InnerVolumeSpecName "kube-api-access-759b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:19.853125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.853075    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca" pod "25a8c176-24e1-458f-ac92-6221ca47e51b" (UID: "25a8c176-24e1-458f-ac92-6221ca47e51b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:19.903195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.903025    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj" pod "5b976102-d3f2-4f24-80ea-bec4ff8610d6" (UID: "5b976102-d3f2-4f24-80ea-bec4ff8610d6"). InnerVolumeSpecName "kube-api-access-nhkpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:19.903195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.903089    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"]
Oct 30 01:30:19.934529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.934491    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a8c176-24e1-458f-ac92-6221ca47e51b-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:19.934529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.934529    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhkpj\" (UniqueName: \"kubernetes.io/projected/5b976102-d3f2-4f24-80ea-bec4ff8610d6-kube-api-access-nhkpj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:19.934810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.934543    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-759b9\" (UniqueName: \"kubernetes.io/projected/25a8c176-24e1-458f-ac92-6221ca47e51b-kube-api-access-759b9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:19.954225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.954198    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:19.958012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.957986    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"]
Oct 30 01:30:19.958790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958767    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:19.958790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958790    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958820    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958829    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958869    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958879    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958890    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958896    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958928    4577 cpu_manager.go:414] "containerMap: removing container" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958935    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958944    4577 cpu_manager.go:414] "containerMap: removing container" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.958985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.958952    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959012    4577 cpu_manager.go:414] "containerMap: removing container" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959022    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959212    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959227    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959238    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959249    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959256    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959269    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959279    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" containerName="cleanup"
Oct 30 01:30:19.959494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.959292    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:19.963123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.963101    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:19.964200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:19.964178    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"]
Oct 30 01:30:20.037101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.037010    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca\") pod \"f00bc966-91f0-47c4-955b-aeace632d6eb\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") "
Oct 30 01:30:20.037287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.037117    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v\" (UniqueName: \"kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v\") pod \"f00bc966-91f0-47c4-955b-aeace632d6eb\" (UID: \"f00bc966-91f0-47c4-955b-aeace632d6eb\") "
Oct 30 01:30:20.037522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.037496    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca" pod "f00bc966-91f0-47c4-955b-aeace632d6eb" (UID: "f00bc966-91f0-47c4-955b-aeace632d6eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:20.037697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.037672    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.037766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.037712    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.037974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.037919    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038004    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038074    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038203    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038249    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038355    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038381    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qslj\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.038509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.038494    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00bc966-91f0-47c4-955b-aeace632d6eb-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.039711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.039683    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v" pod "f00bc966-91f0-47c4-955b-aeace632d6eb" (UID: "f00bc966-91f0-47c4-955b-aeace632d6eb"). InnerVolumeSpecName "kube-api-access-gg96v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:20.139618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139586    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.139618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139635    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139714    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139734    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qslj\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139804    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139832    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.139943    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.140013    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.140074    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.140181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.140180    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg96v\" (UniqueName: \"kubernetes.io/projected/f00bc966-91f0-47c4-955b-aeace632d6eb-kube-api-access-gg96v\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.140981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.140948    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.141126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.141056    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.141585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.141557    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.141895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.141831    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.142155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.142137    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.143630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.143596    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.143909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.143890    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.165294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.165255    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.165621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.165603    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qslj\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj\") pod \"maas-default-gateway-openshift-default-fcb75c45f-rpgs9\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.296544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.296442    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:20.304456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.304434    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"
Oct 30 01:30:20.304592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.304474    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg" event={"ID":"f00bc966-91f0-47c4-955b-aeace632d6eb","Type":"ContainerDied","Data":"6e77a86ab5903fe3ecb83df5c987c396c3664743d86dc7af8622d5da64ff388b"}
Oct 30 01:30:20.304592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.304567    4577 scope.go:122] "RemoveContainer" containerID="8304312b22f576c88348e10114980a83b8d753e96a0ecc0dd0ac3d431bfda917"
Oct 30 01:30:20.306639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.306133    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"
Oct 30 01:30:20.306639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.306137    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf" event={"ID":"5b976102-d3f2-4f24-80ea-bec4ff8610d6","Type":"ContainerDied","Data":"1e5d800b6ca06770f9172014f2a589fb3b6f9f78b3c9dac8692bea00a7b0b078"}
Oct 30 01:30:20.308856 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.308814    4577 generic.go:356] "Generic (PLEG): container finished" podID="01201aae-0a83-48c8-9232-45278008a716" containerID="4ef0769f14f67bd11fa83a1947a77adba573561998b962a637c48a7b1391a1f6" exitCode=60
Oct 30 01:30:20.308981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.308902    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerDied","Data":"4ef0769f14f67bd11fa83a1947a77adba573561998b962a637c48a7b1391a1f6"}
Oct 30 01:30:20.309896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.309871    4577 scope.go:122] "RemoveContainer" containerID="4ef0769f14f67bd11fa83a1947a77adba573561998b962a637c48a7b1391a1f6"
Oct 30 01:30:20.310285 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:20.310261    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w_odh-ai-gateway-infra(01201aae-0a83-48c8-9232-45278008a716)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" podUID="01201aae-0a83-48c8-9232-45278008a716"
Oct 30 01:30:20.312030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.312015    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"
Oct 30 01:30:20.312139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.312058    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf" event={"ID":"25a8c176-24e1-458f-ac92-6221ca47e51b","Type":"ContainerDied","Data":"f75a6c8627e947cd70dce31ecce83239eacac19d879dafb2cb90edb5e90240a1"}
Oct 30 01:30:20.320614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.320592    4577 scope.go:122] "RemoveContainer" containerID="b0edb222ac5de74d12a53f6e820c61cd17c511faf41174bb74ebdfa8dfc31a3f"
Oct 30 01:30:20.338307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.338214    4577 scope.go:122] "RemoveContainer" containerID="886a9075ea4abffccd4fdd2d1a3bb48b1929985b70d590d12a5940cfb568a752"
Oct 30 01:30:20.402981 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.402769    4577 scope.go:122] "RemoveContainer" containerID="31a04a9346c2675fc679f058f53094fa400af87b11c0fb69074d131c118f78ae"
Oct 30 01:30:20.426099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.426071    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"]
Oct 30 01:30:20.435295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.435007    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-df4db1-29888730-l8mpf"]
Oct 30 01:30:20.527779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.527238    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"]
Oct 30 01:30:20.552569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.552463    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-d5b2b8-29888730-q9cmf"]
Oct 30 01:30:20.579338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.579291    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"]
Oct 30 01:30:20.630979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.630938    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888730-49dlg"]
Oct 30 01:30:20.664240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.661299    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"]
Oct 30 01:30:20.672463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.670807    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:20.725555 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.725531    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:20.727587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.727559    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:20.765067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.765032    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca\") pod \"65a8c9e3-563c-4581-841c-c9990a08665e\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") "
Oct 30 01:30:20.765268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.765185    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca\") pod \"184798c0-e246-4b22-b007-e78e7318cf8c\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") "
Oct 30 01:30:20.765268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.765239    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca\") pod \"1230d7f0-c1f2-4674-9a06-e360ae131d59\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") "
Oct 30 01:30:20.765426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.765397    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx\" (UniqueName: \"kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx\") pod \"65a8c9e3-563c-4581-841c-c9990a08665e\" (UID: \"65a8c9e3-563c-4581-841c-c9990a08665e\") "
Oct 30 01:30:20.765488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.765472    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb\" (UniqueName: \"kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb\") pod \"1230d7f0-c1f2-4674-9a06-e360ae131d59\" (UID: \"1230d7f0-c1f2-4674-9a06-e360ae131d59\") "
Oct 30 01:30:20.765553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.765537    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps\" (UniqueName: \"kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps\") pod \"184798c0-e246-4b22-b007-e78e7318cf8c\" (UID: \"184798c0-e246-4b22-b007-e78e7318cf8c\") "
Oct 30 01:30:20.766903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.766877    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca" pod "1230d7f0-c1f2-4674-9a06-e360ae131d59" (UID: "1230d7f0-c1f2-4674-9a06-e360ae131d59"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:20.767326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.767298    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca" pod "184798c0-e246-4b22-b007-e78e7318cf8c" (UID: "184798c0-e246-4b22-b007-e78e7318cf8c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:20.767573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.767550    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca" pod "65a8c9e3-563c-4581-841c-c9990a08665e" (UID: "65a8c9e3-563c-4581-841c-c9990a08665e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:20.769923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.769903    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx" pod "65a8c9e3-563c-4581-841c-c9990a08665e" (UID: "65a8c9e3-563c-4581-841c-c9990a08665e"). InnerVolumeSpecName "kube-api-access-7kccx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:20.772078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.772052    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb" pod "1230d7f0-c1f2-4674-9a06-e360ae131d59" (UID: "1230d7f0-c1f2-4674-9a06-e360ae131d59"). InnerVolumeSpecName "kube-api-access-b5lnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:20.772636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.772619    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps" pod "184798c0-e246-4b22-b007-e78e7318cf8c" (UID: "184798c0-e246-4b22-b007-e78e7318cf8c"). InnerVolumeSpecName "kube-api-access-htsps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:20.868352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.868257    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65a8c9e3-563c-4581-841c-c9990a08665e-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.868352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.868289    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/184798c0-e246-4b22-b007-e78e7318cf8c-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.868352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.868303    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1230d7f0-c1f2-4674-9a06-e360ae131d59-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.868352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.868312    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kccx\" (UniqueName: \"kubernetes.io/projected/65a8c9e3-563c-4581-841c-c9990a08665e-kube-api-access-7kccx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.868352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.868322    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5lnb\" (UniqueName: \"kubernetes.io/projected/1230d7f0-c1f2-4674-9a06-e360ae131d59-kube-api-access-b5lnb\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.868352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.868330    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htsps\" (UniqueName: \"kubernetes.io/projected/184798c0-e246-4b22-b007-e78e7318cf8c-kube-api-access-htsps\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:20.893759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.893538    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:20.893759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.893635    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:20.893759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.893666    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:20.894361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:20.894337    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"]
Oct 30 01:30:21.337169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.337124    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"
Oct 30 01:30:21.337748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.337165    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4" event={"ID":"1230d7f0-c1f2-4674-9a06-e360ae131d59","Type":"ContainerDied","Data":"5fdc55e78e986e8d0d1a1d1bcc3749db4ee78ca5fc29d8aa1fbb1ade1b0421fd"}
Oct 30 01:30:21.337748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.337227    4577 scope.go:122] "RemoveContainer" containerID="dc5a1b76bd3b4a3dca91dd1bd05eb4454d9c750b46f407a595ce791355e1b02d"
Oct 30 01:30:21.342435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.342397    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98" event={"ID":"184798c0-e246-4b22-b007-e78e7318cf8c","Type":"ContainerDied","Data":"8258766a5743e0afe8a7139989b6a7279c1f1c7ea2daabef915d3a9833efa13a"}
Oct 30 01:30:21.342541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.342516    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"
Oct 30 01:30:21.366124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.349200    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" event={"ID":"ba96a3d3-146e-4de3-b926-57f2a456224c","Type":"ContainerStarted","Data":"b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd"}
Oct 30 01:30:21.366124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.349244    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" event={"ID":"ba96a3d3-146e-4de3-b926-57f2a456224c","Type":"ContainerStarted","Data":"64f1a05acc4f436aeed6f9e203f15ffea2b0e6444647562006f10a1eb3b1851d"}
Oct 30 01:30:21.366124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.350820    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"
Oct 30 01:30:21.366124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.350868    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw" event={"ID":"65a8c9e3-563c-4581-841c-c9990a08665e","Type":"ContainerDied","Data":"453ace4875a6e70b05b5180c4d93f1b1bb6eeee3f08a123d82cb66f98e2f6ceb"}
Oct 30 01:30:21.391982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.390324    4577 scope.go:122] "RemoveContainer" containerID="133cad569ac44fce62d9e0a3a3b37fd3299d6e34f538af3e69262b768acfb5a7"
Oct 30 01:30:21.407889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.407836    4577 scope.go:122] "RemoveContainer" containerID="96380703dbf87a1b179052d642d62e9a9ef5d11fd1acb34d0520e7b6eb8b4a3e"
Oct 30 01:30:21.480879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.479363    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"]
Oct 30 01:30:21.493608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.491837    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-72ed13-29888730-ptkh4"]
Oct 30 01:30:21.512661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.512624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:21.533222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.533167    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podStartSLOduration=2.5331495520000002 podStartE2EDuration="2.533149552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:21.528818165 +0000 UTC m=+2929.558128400" watchObservedRunningTime="2026-10-30 01:30:21.533149552 +0000 UTC m=+2929.562459786"
Oct 30 01:30:21.620697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.620411    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:21.620697 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.620513    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"]
Oct 30 01:30:21.624782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.624761    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-c7bba9-29888730-27qbw"]
Oct 30 01:30:21.640401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.640374    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"]
Oct 30 01:30:21.645079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.645054    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-ae1820-29888730-2qm98"]
Oct 30 01:30:21.799772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.799743    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca\") pod \"01201aae-0a83-48c8-9232-45278008a716\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") "
Oct 30 01:30:21.800073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.799891    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw\" (UniqueName: \"kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw\") pod \"01201aae-0a83-48c8-9232-45278008a716\" (UID: \"01201aae-0a83-48c8-9232-45278008a716\") "
Oct 30 01:30:21.800174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.800140    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca" pod "01201aae-0a83-48c8-9232-45278008a716" (UID: "01201aae-0a83-48c8-9232-45278008a716"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:30:21.800769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.800644    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01201aae-0a83-48c8-9232-45278008a716-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:21.802756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.802732    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw" pod "01201aae-0a83-48c8-9232-45278008a716" (UID: "01201aae-0a83-48c8-9232-45278008a716"). InnerVolumeSpecName "kube-api-access-zphhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:30:21.873795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.873676    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:21.902141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.902110    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zphhw\" (UniqueName: \"kubernetes.io/projected/01201aae-0a83-48c8-9232-45278008a716-kube-api-access-zphhw\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:30:21.995165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:21.995127    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:22.119409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.119371    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" path="/var/lib/kubelet/pods/1230d7f0-c1f2-4674-9a06-e360ae131d59/volumes"
Oct 30 01:30:22.120066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.120041    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" path="/var/lib/kubelet/pods/184798c0-e246-4b22-b007-e78e7318cf8c/volumes"
Oct 30 01:30:22.120520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.120501    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25a8c176-24e1-458f-ac92-6221ca47e51b" path="/var/lib/kubelet/pods/25a8c176-24e1-458f-ac92-6221ca47e51b/volumes"
Oct 30 01:30:22.121252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.121228    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" path="/var/lib/kubelet/pods/5b976102-d3f2-4f24-80ea-bec4ff8610d6/volumes"
Oct 30 01:30:22.121714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.121694    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" path="/var/lib/kubelet/pods/65a8c9e3-563c-4581-841c-c9990a08665e/volumes"
Oct 30 01:30:22.122285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.122268    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" path="/var/lib/kubelet/pods/f00bc966-91f0-47c4-955b-aeace632d6eb/volumes"
Oct 30 01:30:22.168870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.168039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:22.311894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.311031    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:22.312667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.312641    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:22.312881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.312839    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:22.393004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.392961    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w" event={"ID":"01201aae-0a83-48c8-9232-45278008a716","Type":"ContainerDied","Data":"e3fd8bff37cc2dc41cede0d2191250e9ec9a40661e9a0dd45c2359db2bbd82b5"}
Oct 30 01:30:22.393004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.392988    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"
Oct 30 01:30:22.393589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.393038    4577 scope.go:122] "RemoveContainer" containerID="4ef0769f14f67bd11fa83a1947a77adba573561998b962a637c48a7b1391a1f6"
Oct 30 01:30:22.477928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.477136    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"]
Oct 30 01:30:22.484054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.483969    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888730-wr85w"]
Oct 30 01:30:22.492719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:22.492678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:23.184440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:23.184358    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:23.297729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:23.297690    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:23.297979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:23.297750    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:23.320417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:23.320378    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:23.810879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:23.809351    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:24.112799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:24.112700    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01201aae-0a83-48c8-9232-45278008a716" path="/var/lib/kubelet/pods/01201aae-0a83-48c8-9232-45278008a716/volumes"
Oct 30 01:30:24.238870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:24.233378    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:24.310937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:24.308139    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:24.310937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:24.308210    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:24.356872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:24.356672    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:25.282087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:25.282034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:25.296827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:25.296790    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:25.297053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:25.296865    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:25.677963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:25.677830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:26.297047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:26.296979    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:26.297047 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:26.297032    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:26.572622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:26.572507    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:27.016495 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:27.016456    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:30:27.303744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:27.303274    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:27.303744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:27.303348    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:27.518317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:27.518273    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:28.297518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:28.297482    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:28.297723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:28.297553    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:28.708023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:28.707936    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:29.296937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:29.296901    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-rpgs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused" start-of-body=
Oct 30 01:30:29.297107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:29.296957    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.248:15021/healthz/ready\": dial tcp 10.134.0.248:15021: connect: connection refused"
Oct 30 01:30:29.338483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:29.338443    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:31.098207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:31.098171    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:31.099641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:31.099617    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:31.100620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:31.100600    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:30:31.123664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:31.123627    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:32.923965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:32.923926    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:33.396131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:33.396080    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:33.533194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:33.533143    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:34.805342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:34.805296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:35.212660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:35.212547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:35.338884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:35.338823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:35.468786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:35.468678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:36.623167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:36.623133    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:38.512618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:38.512564    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:40.171297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:40.171266    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:41.768839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:41.768803    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:41.896580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:41.896513    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:42.043959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:42.043858    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:42.252908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:42.252874    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:42.555322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:42.555287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:42.713183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:42.713149    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:42.916919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:42.916196    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:43.662385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:43.662348    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:43.821961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:43.821932    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:44.782294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:44.782261    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:45.579415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:45.579379    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:47.428905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:47.428840    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:48.974764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:48.974723    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:49.092475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.092440    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:49.262019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.261984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:49.914200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914166    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"]
Oct 30 01:30:49.914929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914914    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:30:49.914929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914929    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914939    4577 cpu_manager.go:414] "containerMap: removing container" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914944    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914954    4577 cpu_manager.go:414] "containerMap: removing container" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914960    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914977    4577 cpu_manager.go:414] "containerMap: removing container" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.914982    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915000    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915004    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915011    4577 cpu_manager.go:414] "containerMap: removing container" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915016    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915028    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915034    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915043    4577 cpu_manager.go:414] "containerMap: removing container" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915047    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915057    4577 cpu_manager.go:414] "containerMap: removing container" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915061    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915069    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:49.915075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915073    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b976102-d3f2-4f24-80ea-bec4ff8610d6" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915165    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915172    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915179    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915186    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915192    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915200    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f00bc966-91f0-47c4-955b-aeace632d6eb" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915205    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915213    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915219    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915224    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:30:49.916045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.915232    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:30:49.921112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.921091    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:49.949916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:49.947971    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"]
Oct 30 01:30:50.015058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015021    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015149    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015189    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015226    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmmhf\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015319    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015402    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015473    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015509    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.015717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.015641    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.116705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.116667    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.116914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.116721    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.116914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.116895    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.116991    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117080    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117095    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117131    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117181    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmmhf\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117231    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117279    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117340    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117456    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.117602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.117542    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.118229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.118201    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.119825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.119802    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.120048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.120031    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.142424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.142398    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.143320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.143297    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmmhf\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf\") pod \"maas-default-gateway-openshift-default-fcb75c45f-9r8sf\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.265280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.265181    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:50.733598 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:30:50.733548    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa83ebe_aaad_4b21_9c86_35b4b2eaaf25.slice/crio-52e66468ef226ce58439f0604082a14602ae41368d1efd06f1f5b50ef9022ef1 WatchSource:0}: Error finding container 52e66468ef226ce58439f0604082a14602ae41368d1efd06f1f5b50ef9022ef1: Status 404 returned error can't find the container with id 52e66468ef226ce58439f0604082a14602ae41368d1efd06f1f5b50ef9022ef1
Oct 30 01:30:50.733755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.733603    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"]
Oct 30 01:30:50.736712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.736679    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:50.736875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.736760    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:50.736875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.736804    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:30:50.964273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:50.964239    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:51.194836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:51.194741    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:51.275242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:51.275202    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" event={"ID":"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25","Type":"ContainerStarted","Data":"89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426"}
Oct 30 01:30:51.275450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:51.275270    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" event={"ID":"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25","Type":"ContainerStarted","Data":"52e66468ef226ce58439f0604082a14602ae41368d1efd06f1f5b50ef9022ef1"}
Oct 30 01:30:51.377383 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:51.377308    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podStartSLOduration=2.377286042 podStartE2EDuration="2.377286042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:51.342696823 +0000 UTC m=+2959.372007053" watchObservedRunningTime="2026-10-30 01:30:51.377286042 +0000 UTC m=+2959.406596273"
Oct 30 01:30:51.900988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:51.900956    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:52.274140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:52.269020    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:30:52.372881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:52.357985    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:52.372881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:52.358043    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:53.092470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:53.092422    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:53.275370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:53.275331    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:53.276075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:53.276044    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:53.984540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:53.984503    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:54.266372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:54.266273    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:54.266662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:54.266631    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:54.580487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:54.580386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:54.691560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:54.691521    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:55.064235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:55.064186    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:55.267317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:55.267280    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:55.267707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:55.267656    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:56.273868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:56.273335    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:56.273868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:56.273411    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:56.583671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:56.583568    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:56.915200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:56.915115    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:57.225305 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:30:57.225209    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:30:57.266330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:57.266290    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:57.266633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:57.266601    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:57.580157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:57.576267    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:58.266657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:58.266624    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:58.266890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:58.266678    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:30:58.711626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:58.711537    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:30:59.265936 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:59.265901    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-9r8sf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused" start-of-body=
Oct 30 01:30:59.266132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:30:59.265958    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.249:15021/healthz/ready\": dial tcp 10.134.0.249:15021: connect: connection refused"
Oct 30 01:31:00.409652 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:00.409611    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:00.927176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:00.927139    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:31:00.927750 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:00.927731    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:31:00.929001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:00.928977    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:31:01.417484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:01.417452    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:01.530649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:01.530612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:02.252492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:02.252456    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:03.685165 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.685129    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"]
Oct 30 01:31:03.686405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686377    4577 cpu_manager.go:414] "containerMap: removing container" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:31:03.686405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686403    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01201aae-0a83-48c8-9232-45278008a716" containerName="cleanup"
Oct 30 01:31:03.686622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686425    4577 cpu_manager.go:414] "containerMap: removing container" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:31:03.686622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686434    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:31:03.686622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686467    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:31:03.686622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686475    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:31:03.686622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686494    4577 cpu_manager.go:414] "containerMap: removing container" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:31:03.686622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686500    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184798c0-e246-4b22-b007-e78e7318cf8c" containerName="cleanup"
Oct 30 01:31:03.686968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686690    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="65a8c9e3-563c-4581-841c-c9990a08665e" containerName="cleanup"
Oct 30 01:31:03.686968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.686701    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1230d7f0-c1f2-4674-9a06-e360ae131d59" containerName="cleanup"
Oct 30 01:31:03.691706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.691683    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.713006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.706228    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"]
Oct 30 01:31:03.714977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.714442    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"payload-processing-external-model-e2e--42876bb9-dockercfg-snpkn\""
Oct 30 01:31:03.714977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.714570    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"agc-cred-6fb94fa6b5f6\""
Oct 30 01:31:03.714977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.714728    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"openshift-service-ca.crt\""
Oct 30 01:31:03.714977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.714765    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"payload-processing-external-model-plugins-e2e-extmod-a-c7bba9\""
Oct 30 01:31:03.714977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.714825    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"kube-root-ca.crt\""
Oct 30 01:31:03.714977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.714932    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"routing-overlay\""
Oct 30 01:31:03.726242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.726217    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-c7bba9\"/\"agc-cred-71c0954440c8\""
Oct 30 01:31:03.780692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.780654    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg2qj\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.781101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.781080    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.781315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.781300    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.781488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.781472    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.883838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.883801    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg2qj\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.884267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.884242    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.884942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.884911    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.885086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.884991    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.885719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.885695    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.886147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.886121    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.890471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.890450    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:03.909455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:03.909423    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg2qj\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj\") pod \"payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") " pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:04.003857 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:04.003774    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:04.020681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:04.020652    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:04.432708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:04.432679    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"]
Oct 30 01:31:04.433633 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:31:04.433610    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c78d09_634e_4e14_bc28_ac26cdba37dd.slice/crio-1c8f355002bf3923b2ebc42071b28aed064387fd05596e6cba8f582e48f52ecc WatchSource:0}: Error finding container 1c8f355002bf3923b2ebc42071b28aed064387fd05596e6cba8f582e48f52ecc: Status 404 returned error can't find the container with id 1c8f355002bf3923b2ebc42071b28aed064387fd05596e6cba8f582e48f52ecc
Oct 30 01:31:04.479928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:04.479890    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" event={"ID":"f6c78d09-634e-4e14-bc28-ac26cdba37dd","Type":"ContainerStarted","Data":"1c8f355002bf3923b2ebc42071b28aed064387fd05596e6cba8f582e48f52ecc"}
Oct 30 01:31:05.489721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:05.489684    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" event={"ID":"f6c78d09-634e-4e14-bc28-ac26cdba37dd","Type":"ContainerStarted","Data":"15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b"}
Oct 30 01:31:05.490240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:05.489801    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:05.538759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:05.538714    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" podStartSLOduration=2.5387002929999998 podStartE2EDuration="2.538700293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:31:05.534874357 +0000 UTC m=+2973.564184612" watchObservedRunningTime="2026-10-30 01:31:05.538700293 +0000 UTC m=+2973.568010523"
Oct 30 01:31:05.809176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:05.809080    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:07.603870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:07.603805    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:09.286684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:09.286650    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:11.234409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:11.234374    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:13.018181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:13.018144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:15.474871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:15.467431    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:16.370745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:16.370100    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-4x4jk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.247:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:31:16.370745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:16.370183    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4x4jk" podUID="d07ac1e5-f1b2-4018-9cfb-1f71f672c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.247:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:31:16.779193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:16.776934    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:31:16.813777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:16.812874    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:31:17.319324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:17.319277    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:19.099025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:19.098975    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:20.939442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:20.939394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:22.735181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:22.735147    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:24.316815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:24.316779    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:24.507687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:24.507655    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:26.201486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:26.201447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:27.016079 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:31:27.016037    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:31:27.880524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:27.880491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:29.590392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:29.590361    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:30.725785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:30.725746    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:30.848883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:30.848834    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:31.412314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:31.412274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:33.124224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:33.124186    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:34.876775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:34.876739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:36.733028 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:36.732994    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:38.532932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:38.532899    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:39.750705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:39.750670    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:39.909097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:39.909035    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:40.389375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:40.389320    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:40.833703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:40.833664    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:40.974421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:40.974378    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:41.502912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:41.484923    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:42.303868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:42.300270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:44.190100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:44.190067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:46.060448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:46.060410    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:48.299833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:48.299755    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:49.979172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:49.979125    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:50.118819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:50.118776    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:50.239772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:50.239683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:50.426361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:50.426323    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:51.617871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:51.604204    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:31:51.766873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:51.763497    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:31:52.813836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:52.813798    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:54.810544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:54.810509    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:56.823670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:56.823588    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:57.028684 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:31:57.028651    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:31:58.105710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:58.105679    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:58.267933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:58.267885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:58.755765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:58.755725    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:58.977672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:58.977637    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:59.146455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.146354    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"]
Oct 30 01:31:59.151296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.151268    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.178245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.178213    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"]
Oct 30 01:31:59.183654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.183628    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-415c6b-openshift-default-dockercfg-fd59f\""
Oct 30 01:31:59.197426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.197397    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414504    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414548    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414574    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414624    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414651    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414681    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414753    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6697\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414789    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.416921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.414905    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.541614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541555    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.541614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541604    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541635    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541693    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541727    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541761    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541869    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6697\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.541909    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.542029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.542029    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.543435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.542929    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.543435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.543193    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.552902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.550479    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.552902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.551092    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.552902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.551248    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.562582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.562512    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.566950 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.566925    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.580273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.580226    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.580941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.580916    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6697\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697\") pod \"e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") " pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:31:59.755205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.755164    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:31:59.776237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:31:59.776206    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:32:00.122426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.122266    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"]
Oct 30 01:32:00.124779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.124722    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:32:00.124926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.124886    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:32:00.124995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.124970    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:32:00.570557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.570524    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" event={"ID":"434adec4-4a45-460c-a1c8-36cfbdd2092a","Type":"ContainerStarted","Data":"f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"}
Oct 30 01:32:00.571122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.570585    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" event={"ID":"434adec4-4a45-460c-a1c8-36cfbdd2092a","Type":"ContainerStarted","Data":"6b54f3804d94a07fd3cf99439d3c20991c2c6c888ac1a71b6e2bbb65c2e371bf"}
Oct 30 01:32:00.636619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.636564    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podStartSLOduration=1.636549094 podStartE2EDuration="1.636549094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:31:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:32:00.63308373 +0000 UTC m=+3028.662393964" watchObservedRunningTime="2026-10-30 01:32:00.636549094 +0000 UTC m=+3028.665859324"
Oct 30 01:32:00.776557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.776522    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:32:00.785912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:00.785883    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:01.783640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:01.783610    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:32:02.590078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:02.590048    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:32:02.591263 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:02.591239    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:32:02.738382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:02.738351    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:03.139803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:03.139770    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:03.288471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:03.288428    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:04.529281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:04.529250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:04.694872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:04.694400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:04.898925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:04.890711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:05.014589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:05.014519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:05.505882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:05.504287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:06.563701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:06.563659    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:06.808105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:06.808066    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:07.122454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:07.122421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:07.283870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:07.283785    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:08.912871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:08.912043    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:10.720826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:10.720788    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:11.553940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:11.553895    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:11.733438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:11.733402    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:13.049999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:13.049939    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:13.374117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:13.374029    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:13.596208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:13.596175    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:15.315774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:15.315737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:17.036116 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:17.036069    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:18.918472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:18.918418    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:20.687895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:20.687863    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:22.478097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:22.478060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:23.831314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:23.831263    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:23.948473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:23.948432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:24.253958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:24.253924    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:24.464449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:24.464400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:26.028716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:26.028661    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:27.015712 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:32:27.015673    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:32:27.773782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:27.773743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:29.445934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:29.445890    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:31.272478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:31.272422    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:33.143392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:33.143351    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:35.282886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:35.282838    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:37.525982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:37.525935    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:32:37.581829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:37.581702    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:37.755203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:37.753937    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:32:39.421720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:39.421686    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:41.256278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:41.256249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:43.033784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:43.033746    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:45.036507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:45.036478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:46.516235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:46.516194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:46.845628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:46.845517    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:48.875218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:48.875184    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:50.748021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:50.747991    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:52.490498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:52.490459    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:52.655948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:52.625199    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:54.267948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:54.267913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:56.074266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:56.074224    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:57.015791 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:32:57.015752    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:32:57.830908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:57.830873    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:32:59.744866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:32:59.744807    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:01.654120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:01.654085    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:01.768723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:01.768689    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:01.895290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:01.895253    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:03.762321 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:03.762285    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:05.701449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:05.701408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:07.492062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:07.492028    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:09.324093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:09.324058    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:11.117307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:11.117269    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:12.905514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:12.905420    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:14.620304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:14.620268    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:16.306108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:16.306065    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:18.132626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:18.132588    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:19.835209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:19.835178    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:21.625811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:21.625772    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:23.399147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:23.399111    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:24.578968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:24.578930    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:25.084292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:25.084256    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:26.922681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:26.922648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:27.065744 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:33:27.065707    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:33:27.971444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:27.971367    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:28.724198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:28.724169    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:30.645880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:30.640346    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:32.423507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:32.423476    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:34.104446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:34.104401    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:35.764945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:35.764910    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:37.628571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.628536    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:37.787173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.787123    4577 generic.go:356] "Generic (PLEG): container finished" podID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerID="f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c" exitCode=0
Oct 30 01:33:37.787368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.787201    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" event={"ID":"434adec4-4a45-460c-a1c8-36cfbdd2092a","Type":"ContainerDied","Data":"f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"}
Oct 30 01:33:37.787792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.787775    4577 scope.go:122] "RemoveContainer" containerID="f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"
Oct 30 01:33:37.790507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.790472    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:33:37.790593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.790569    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:33:37.790637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:37.790617    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:33:38.799781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:38.799744    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" event={"ID":"434adec4-4a45-460c-a1c8-36cfbdd2092a","Type":"ContainerStarted","Data":"7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a"}
Oct 30 01:33:39.379735 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:39.379371    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:39.777679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:39.777235    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:33:39.777679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:39.777282    4577 patch_prober.go:29] interesting pod/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused" start-of-body=
Oct 30 01:33:39.777679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:39.777354    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused"
Oct 30 01:33:40.777615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:40.777481    4577 patch_prober.go:29] interesting pod/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused" start-of-body=
Oct 30 01:33:40.777615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:40.777563    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused"
Oct 30 01:33:41.167473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:41.167386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:41.780299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:41.780255    4577 patch_prober.go:29] interesting pod/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused" start-of-body=
Oct 30 01:33:41.780838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:41.780339    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused"
Oct 30 01:33:42.780349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:42.780312    4577 patch_prober.go:29] interesting pod/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused" start-of-body=
Oct 30 01:33:42.781012 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:42.780981    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused"
Oct 30 01:33:42.927002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:42.926965    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:43.792364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:43.792301    4577 patch_prober.go:29] interesting pod/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused" start-of-body=
Oct 30 01:33:43.793376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:43.793309    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.251:15021/healthz/ready\": dial tcp 10.134.0.251:15021: connect: connection refused"
Oct 30 01:33:44.687488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:44.687451    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:44.921152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:44.921123    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:33:44.921592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:44.921539    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:33:44.922390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:44.922368    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:33:45.306773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:45.306739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:45.747079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:45.747040    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:46.204562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:46.204475    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:46.706990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:46.706959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:46.826228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:46.826192    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:48.528126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:48.528089    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:50.401878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:50.399368    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:50.602875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:50.598800    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:52.064704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.064674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:52.178155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.178119    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"]
Oct 30 01:33:52.183705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.183675    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.207895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.198407    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:52.210934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.210905    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ab0a2c-dockercfg-4dnfd\""
Oct 30 01:33:52.215701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.215666    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"]
Oct 30 01:33:52.301938 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.293114    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ab0a2c\""
Oct 30 01:33:52.324825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.324737    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7vqp\" (UniqueName: \"kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp\") pod \"payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.339768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.339733    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.444874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.444522    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7vqp\" (UniqueName: \"kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp\") pod \"payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.444874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.444756    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.462870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.446201    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.491866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.487645    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7vqp\" (UniqueName: \"kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp\") pod \"payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.508313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.508279    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:52.605051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.604199    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:52.978484 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:33:52.978452    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda276ce33_eadf_4310_b249_9b1d35c82eeb.slice/crio-71e57518422a2b3b0949ab8e237ec7c445cd749aa66966bebe797c9eb9c7148a WatchSource:0}: Error finding container 71e57518422a2b3b0949ab8e237ec7c445cd749aa66966bebe797c9eb9c7148a: Status 404 returned error can't find the container with id 71e57518422a2b3b0949ab8e237ec7c445cd749aa66966bebe797c9eb9c7148a
Oct 30 01:33:52.980134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:52.980104    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"]
Oct 30 01:33:53.946692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:53.946649    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" event={"ID":"a276ce33-eadf-4310-b249-9b1d35c82eeb","Type":"ContainerStarted","Data":"12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1"}
Oct 30 01:33:53.947154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:53.946765    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:33:53.947154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:53.946786    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" event={"ID":"a276ce33-eadf-4310-b249-9b1d35c82eeb","Type":"ContainerStarted","Data":"71e57518422a2b3b0949ab8e237ec7c445cd749aa66966bebe797c9eb9c7148a"}
Oct 30 01:33:53.990001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:53.989955    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:54.005039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:54.004989    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" podStartSLOduration=2.004967785 podStartE2EDuration="2.004967785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:33:54.001341951 +0000 UTC m=+3142.030652169" watchObservedRunningTime="2026-10-30 01:33:54.004967785 +0000 UTC m=+3142.034278023"
Oct 30 01:33:54.104984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:54.104948    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:54.918524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:54.918489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:56.704693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:56.704656    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:57.021455 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:33:57.021417    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:33:58.544499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:58.544467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:58.948489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:58.948402    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:59.407268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.407231    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:59.813549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.813516    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"]
Oct 30 01:33:59.819635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.819606    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.845893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.845020    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"]
Oct 30 01:33:59.850874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.850835    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-190407-openshift-default-dockercfg-dqpvk\""
Oct 30 01:33:59.866669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.866636    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:33:59.921919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.921778    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922294    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922367    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922426    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922457    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922490    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922556    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.922903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.922789    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvpf6\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:33:59.953873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:33:59.938986    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044403    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvpf6\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044461    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044533    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044744    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044806    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044885    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044915    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.044948    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.045016    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.045749    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.046871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.046381    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.072926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.066052    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.072926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.066325    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.072926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.067416    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.117891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.115917    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.132896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.131293    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.158886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.158754    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvpf6\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.206894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.204949    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token\") pod \"e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") " pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.403397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.403305    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:00.491890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.484463    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:00.520417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.520378    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:00.832941 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:34:00.832912    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7603959c_91b2_462c_81e7_98e0799baebf.slice/crio-abe03530be797a6bc7ad394ae7a23f7d43db53176e3cbfa016da0d8c59aef54e WatchSource:0}: Error finding container abe03530be797a6bc7ad394ae7a23f7d43db53176e3cbfa016da0d8c59aef54e: Status 404 returned error can't find the container with id abe03530be797a6bc7ad394ae7a23f7d43db53176e3cbfa016da0d8c59aef54e
Oct 30 01:34:00.833918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.833814    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"]
Oct 30 01:34:00.836615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.836583    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:34:00.836702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.836651    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:34:00.836702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:00.836683    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:34:01.101726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:01.101616    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" event={"ID":"7603959c-91b2-462c-81e7-98e0799baebf","Type":"ContainerStarted","Data":"2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"}
Oct 30 01:34:01.101726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:01.101686    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" event={"ID":"7603959c-91b2-462c-81e7-98e0799baebf","Type":"ContainerStarted","Data":"abe03530be797a6bc7ad394ae7a23f7d43db53176e3cbfa016da0d8c59aef54e"}
Oct 30 01:34:01.187086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:01.187031    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podStartSLOduration=2.187011197 podStartE2EDuration="2.187011197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:34:01.186538043 +0000 UTC m=+3149.215848275" watchObservedRunningTime="2026-10-30 01:34:01.187011197 +0000 UTC m=+3149.216321430"
Oct 30 01:34:01.484995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:01.484956    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:01.492226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:01.492198    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:02.121371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:02.121323    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:02.121908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:02.121506    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:34:02.223748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:02.223717    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:03.580199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:03.580161    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:03.707196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:03.707158    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:04.186836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:04.186802    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:04.979616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:04.979588    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:34:05.267648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:05.267567    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:05.391186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:05.391149    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:06.096787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:06.096756    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:07.873463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:07.873432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:09.665939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:09.665894    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:11.430081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:11.430044    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:13.105422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:13.105380    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:14.839954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:14.839913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:16.619334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:16.619269    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:16.733833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:16.733790    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:17.069382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:17.069340    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:17.187283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:17.187236    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:17.457233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:17.457119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:17.615247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:17.615204    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:17.726783 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:17.726655    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:18.396500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:18.396457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:18.595732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:18.595689    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:18.718204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:18.718117    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:19.167778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:19.167741    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:19.292474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:19.292437    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:20.199896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:20.199830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:21.408826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:21.408791    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"]
Oct 30 01:34:21.476204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:21.476167    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:21.955723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:21.955683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:23.721221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:23.721182    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:24.218940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:24.218908    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:24.347050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:24.347009    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:24.682088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:24.682050    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:24.793018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:24.792980    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:25.130303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.130263    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"]
Oct 30 01:34:25.139857 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.139827    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.163590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.163553    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"]
Oct 30 01:34:25.164766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.164718    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.164995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.164971    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.165099 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.165011    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.165176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.165114    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.165296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.165274    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.165400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.165318    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.166040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.166010    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.166175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.166060    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dh75\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.166175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.166126    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.184643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.184601    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-db4dd2-openshift-default-dockercfg-ht87t\""
Oct 30 01:34:25.185379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.185271    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:25.267145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267114    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.267350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267226    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.267350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267262    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.267350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267315    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.267550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267524    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dh75\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.267655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267618    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.267888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267679    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.268200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.267723    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.268301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.268280    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.268365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.268027    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.268483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.268466    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.268534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.268501    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.268868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.268824    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.269269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.269248    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.269918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.269893    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.270389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.270368    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.332873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.308092    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.332873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.330052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:25.388915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.387496    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dh75\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75\") pod \"e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") " pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.482439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.482364    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:25.552508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.552458    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:25.871892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.865645    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:25.995873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:25.992276    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:26.230581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.230538    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"]
Oct 30 01:34:26.233339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.233289    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:34:26.233494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.233404    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:34:26.233494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.233448    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:34:26.324740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.324703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:26.409760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.409719    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" podUID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" containerName="payload-processing" containerID="cri-o://15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b" gracePeriod=25
Oct 30 01:34:26.479658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.479626    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" event={"ID":"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460","Type":"ContainerStarted","Data":"a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"}
Oct 30 01:34:26.479877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.479679    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" event={"ID":"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460","Type":"ContainerStarted","Data":"b7f937b1ef3efb7601dd7edb70ba4e5fd15201c9362c269d7916f712ce406dd4"}
Oct 30 01:34:26.483714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.483691    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:26.498571 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:34:26.498543    4577 logging.go:55] [core] [Channel #7007 SubChannel #7008]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.250:50052", ServerName: "10.134.0.250:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.250:50052: connect: connection refused"
Oct 30 01:34:26.536463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.536403    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podStartSLOduration=1.5363821519999998 podStartE2EDuration="1.536382152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:34:26.528430839 +0000 UTC m=+3174.557741086" watchObservedRunningTime="2026-10-30 01:34:26.536382152 +0000 UTC m=+3174.565692384"
Oct 30 01:34:26.774392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.774027    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:34:26.801658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.801535    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:26.951925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.938386    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials\") pod \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") "
Oct 30 01:34:26.951925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.938435    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj\") pod \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") "
Oct 30 01:34:26.951925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.938897    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume\") pod \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") "
Oct 30 01:34:26.951925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.938975    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume\") pod \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\" (UID: \"f6c78d09-634e-4e14-bc28-ac26cdba37dd\") "
Oct 30 01:34:26.951925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.950230    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume" pod "f6c78d09-634e-4e14-bc28-ac26cdba37dd" (UID: "f6c78d09-634e-4e14-bc28-ac26cdba37dd"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:34:26.957217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:26.957180    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume" pod "f6c78d09-634e-4e14-bc28-ac26cdba37dd" (UID: "f6c78d09-634e-4e14-bc28-ac26cdba37dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:34:27.099530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.099444    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj" pod "f6c78d09-634e-4e14-bc28-ac26cdba37dd" (UID: "f6c78d09-634e-4e14-bc28-ac26cdba37dd"). InnerVolumeSpecName "kube-api-access-fg2qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:34:27.257974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.117404    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:34:27.258193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.257997    4577 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f6c78d09-634e-4e14-bc28-ac26cdba37dd-routing-overlay-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:34:27.277777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.277674    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials" pod "f6c78d09-634e-4e14-bc28-ac26cdba37dd" (UID: "f6c78d09-634e-4e14-bc28-ac26cdba37dd"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:34:27.300218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.300185    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:27.313342 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:34:27.313303    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:34:27.362785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.362693    4577 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-provider-credentials\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:34:27.363041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.363019    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg2qj\" (UniqueName: \"kubernetes.io/projected/f6c78d09-634e-4e14-bc28-ac26cdba37dd-kube-api-access-fg2qj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:34:27.492501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.492463    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:27.492969 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.492937    4577 generic.go:356] "Generic (PLEG): container finished" podID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" containerID="15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b" exitCode=0
Oct 30 01:34:27.493111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.492990    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"
Oct 30 01:34:27.493111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.493019    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" event={"ID":"f6c78d09-634e-4e14-bc28-ac26cdba37dd","Type":"ContainerDied","Data":"15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b"}
Oct 30 01:34:27.493111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.493069    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" event={"ID":"f6c78d09-634e-4e14-bc28-ac26cdba37dd","Type":"ContainerDied","Data":"1c8f355002bf3923b2ebc42071b28aed064387fd05596e6cba8f582e48f52ecc"}
Oct 30 01:34:27.493111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.493091    4577 scope.go:122] "RemoveContainer" containerID="15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b"
Oct 30 01:34:27.493590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.493487    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:27.494759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.494641    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:34:27.498337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.498296    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2" podUID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.250:50052\" within 1s: context deadline exceeded"
Oct 30 01:34:27.505206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.505190    4577 scope.go:122] "RemoveContainer" containerID="15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b"
Oct 30 01:34:27.505473 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:34:27.505453    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b\": container with ID starting with 15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b not found: ID does not exist" containerID="15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b"
Oct 30 01:34:27.505543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.505483    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b"} err="failed to get container status \"15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b\": rpc error: code = NotFound desc = could not find container \"15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b\": container with ID starting with 15cb2c12af9258894d9fa7daf2fd56b08524c5303b4332fc7a87e935f32c1f8b not found: ID does not exist"
Oct 30 01:34:27.626964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.623981    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"]
Oct 30 01:34:27.629516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:27.629486    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-c7bba9/payload-processing-external-model-e2e-extmod-a-c7bba9-6879fl6v2"]
Oct 30 01:34:28.113302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:28.113271    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" path="/var/lib/kubelet/pods/f6c78d09-634e-4e14-bc28-ac26cdba37dd/volumes"
Oct 30 01:34:28.674682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:28.674643    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:29.204363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:29.204324    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:29.346117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:29.346075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:29.494869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:29.481105    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:30.895595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:30.895561    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:31.010583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:31.010547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:31.348186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:31.346937    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:31.495477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:31.495386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:32.860690 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:32.860657    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:32.975234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:32.975198    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:33.305516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:33.305473    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:34.987734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:34.987691    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:36.713744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:36.713711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:38.376518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:38.376485    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:40.122311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:40.122279    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:41.868606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:41.868505    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:41.982126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:41.982081    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:42.101646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:42.101608    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:43.661739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:43.661641    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:45.528235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:45.528198    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:47.201151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:47.201113    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:48.939696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:48.939661    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:50.722924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:50.722885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:52.394587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:52.394550    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:54.200218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:54.200177    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:54.738987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:54.738939    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:34:54.764182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:54.764139    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:34:55.034929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:55.033146    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:34:55.971357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:55.971323    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:56.095066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:56.094995    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:57.015919 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:34:57.015882    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:34:57.892063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:57.892030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:34:59.558427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:34:59.558396    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:01.316777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:01.316743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:03.096074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:03.096031    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:04.850424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:04.850385    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:06.610706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:06.610670    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:08.392137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:08.392087    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:10.190479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:10.190419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:11.921200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:11.921166    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:13.572307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:13.572262    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:15.264641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:15.264607    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:17.126416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:17.126382    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:18.901833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:18.901785    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:20.664868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:20.664445    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:22.389994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.389954    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"]
Oct 30 01:35:22.391208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.391180    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" containerName="payload-processing"
Oct 30 01:35:22.391208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.391211    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" containerName="payload-processing"
Oct 30 01:35:22.391440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.391427    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6c78d09-634e-4e14-bc28-ac26cdba37dd" containerName="payload-processing"
Oct 30 01:35:22.415186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.415159    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"]
Oct 30 01:35:22.415448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.415437    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.433197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.433171    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-db4dd2-serving-cert\""
Oct 30 01:35:22.484444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.484392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:22.590029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.572255    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xmv7\" (UniqueName: \"kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.615609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.615576    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.616336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.616300    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.617220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.617196    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.617714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.617069    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:22.718882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.718768    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xmv7\" (UniqueName: \"kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.718882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.718823    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.719137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.719077    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.719137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.719108    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.719942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.719915    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.722355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.722332    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.722501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.722482    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:22.744069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:22.744026    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xmv7\" (UniqueName: \"kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7\") pod \"maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:23.037048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:23.037017    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:23.388083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:23.388049    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"]
Oct 30 01:35:23.388232 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:35:23.388046    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b305fa1_a45e_4810_810b_e3f661117406.slice/crio-75494021dafdcc6e7471eaf508b2ee8957c588994c3efc6096fbf978c1c918b2 WatchSource:0}: Error finding container 75494021dafdcc6e7471eaf508b2ee8957c588994c3efc6096fbf978c1c918b2: Status 404 returned error can't find the container with id 75494021dafdcc6e7471eaf508b2ee8957c588994c3efc6096fbf978c1c918b2
Oct 30 01:35:23.390026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:23.390004    4577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:35:24.322882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:24.322836    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:24.399872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:24.394106    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q" event={"ID":"8b305fa1-a45e-4810-810b-e3f661117406","Type":"ContainerStarted","Data":"a665b176589ed362951645562ae9f15182ec40e4f335dca53e100e698a7dad4c"}
Oct 30 01:35:24.399872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:24.394197    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q" event={"ID":"8b305fa1-a45e-4810-810b-e3f661117406","Type":"ContainerStarted","Data":"75494021dafdcc6e7471eaf508b2ee8957c588994c3efc6096fbf978c1c918b2"}
Oct 30 01:35:24.403671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:24.403639    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:24.478149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:24.478088    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q" podStartSLOduration=1.9677307769999999 podStartE2EDuration="2.478070893s" totalImagesPullingTime="510.340116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:35:24.466900669 +0000 UTC m=+3232.496210933" watchObservedRunningTime="2026-10-30 01:35:24.478070893 +0000 UTC m=+3232.507381125"
Oct 30 01:35:24.834453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:24.834416    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:26.074202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:26.074155    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:26.189672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:26.189631    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:27.015693 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:35:27.015658    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:35:27.803027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:27.802970    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:29.659874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:29.659817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:30.449051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:30.449019    4577 generic.go:356] "Generic (PLEG): container finished" podID="7603959c-91b2-462c-81e7-98e0799baebf" containerID="2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1" exitCode=0
Oct 30 01:35:30.449289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:30.449094    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" event={"ID":"7603959c-91b2-462c-81e7-98e0799baebf","Type":"ContainerDied","Data":"2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"}
Oct 30 01:35:30.449613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:30.449597    4577 scope.go:122] "RemoveContainer" containerID="2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"
Oct 30 01:35:30.451781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:30.451746    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:35:30.451914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:30.451828    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:35:30.451914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:30.451893    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:35:31.401126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:31.401082    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:31.416977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:31.416951    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:35:31.542872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:31.533826    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" event={"ID":"7603959c-91b2-462c-81e7-98e0799baebf","Type":"ContainerStarted","Data":"c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308"}
Oct 30 01:35:32.115523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:32.115498    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:35:32.484930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:32.484879    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:35:32.486575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:32.486548    4577 patch_prober.go:29] interesting pod/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 01:35:32.486751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:32.486606    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 01:35:33.130877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:33.130829    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:33.485032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:33.484996    4577 patch_prober.go:29] interesting pod/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 01:35:33.485681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:33.485061    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 01:35:34.485196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:34.485162    4577 patch_prober.go:29] interesting pod/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 01:35:34.485616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:34.485213    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 01:35:34.872974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:34.872816    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:35.484972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:35.484931    4577 patch_prober.go:29] interesting pod/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 01:35:35.485211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:35.484992    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 01:35:36.485703 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:36.485671    4577 patch_prober.go:29] interesting pod/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused" start-of-body=
Oct 30 01:35:36.486125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:36.485729    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.253:15021/healthz/ready\": dial tcp 10.134.0.253:15021: connect: connection refused"
Oct 30 01:35:36.663375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:36.663328    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:37.491438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:37.491407    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:35:37.492472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:37.492450    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:35:38.378863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:38.378811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:40.061870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:40.056740    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:41.938178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:41.938147    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:42.804641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:42.804605    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:43.702612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:43.702569    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:44.714716 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:44.714675    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:45.418205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:45.418176    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:45.563868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:45.558893    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:45.672389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:45.672305    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:46.468198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:46.468148    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:46.590758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:46.590720    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:47.512282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:47.512248    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:49.298008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:49.297974    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:51.019140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:51.019109    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:52.896357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:52.896316    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:54.630435 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:54.630357    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:56.211155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:56.211123    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:56.496303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:56.496274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:35:57.016122 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:35:57.016086    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:35:58.277182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:35:58.277144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:00.067573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:00.067530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:01.830888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:01.830834    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:02.480332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:02.480297    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:03.705960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:03.705921    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:05.503760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:05.503722    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:07.243630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:07.243591    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:08.195959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:08.195915    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:08.456300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:08.456209    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:08.596163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:08.596132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:09.084870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:09.078522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:09.193094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:09.193046    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:10.677140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:10.677108    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:10.844260 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:10.844222    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:12.626799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:12.626711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:13.197258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:13.197223    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:13.550894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:13.544935    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:36:13.550894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:13.545000    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:36:14.153211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.153109    4577 generic.go:356] "Generic (PLEG): container finished" podID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerID="a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d" exitCode=0
Oct 30 01:36:14.153211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.153184    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" event={"ID":"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460","Type":"ContainerDied","Data":"a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"}
Oct 30 01:36:14.153826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.153714    4577 scope.go:122] "RemoveContainer" containerID="a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"
Oct 30 01:36:14.156205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.156170    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:14.156302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.156262    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:14.156352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.156326    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:14.658160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:14.658120    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:15.164011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:15.163966    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" event={"ID":"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460","Type":"ContainerStarted","Data":"6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb"}
Oct 30 01:36:15.483318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:15.483273    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:36:15.483548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:15.483515    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused" start-of-body=
Oct 30 01:36:15.483631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:15.483573    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused"
Oct 30 01:36:15.789225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:15.789137    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:15.908440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:15.908402    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:16.268810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.268772    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:16.382641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.382608    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:16.488029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.487995    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused" start-of-body=
Oct 30 01:36:16.488346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.488313    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused"
Oct 30 01:36:16.697676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.697589    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"]
Oct 30 01:36:16.707385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.707174    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.739820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.739781    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"]
Oct 30 01:36:16.773937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.773903    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6aa9aa-openshift-default-dockercfg-wrslw\""
Oct 30 01:36:16.775793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.775762    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:16.836825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.836795    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837135    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837251    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837361    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837457    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837600    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837754    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.837956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.837925    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.838256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.838219    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqxcb\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942120    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942182    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942239    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942409    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqxcb\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942555    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942617    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942652    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942691    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.942741    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.948920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.944162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.950518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.950487    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:16.951000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.950974    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.005038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.990671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:17.005038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:16.991375    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.021430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.021392    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.022014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.021988    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.030825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.030507    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.093460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.093423    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.150751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.150713    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqxcb\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb\") pod \"e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") " pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.328898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.327415    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:17.488112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.487605    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused" start-of-body=
Oct 30 01:36:17.488112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.487669    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused"
Oct 30 01:36:17.701906 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:36:17.701875    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78892cf_9422_4937_b888_0e7d5326fd13.slice/crio-c9c577580a900a2da7f1a78a4765879d6ec453f36c46e5a68bf4f8204dbc7d7b WatchSource:0}: Error finding container c9c577580a900a2da7f1a78a4765879d6ec453f36c46e5a68bf4f8204dbc7d7b: Status 404 returned error can't find the container with id c9c577580a900a2da7f1a78a4765879d6ec453f36c46e5a68bf4f8204dbc7d7b
Oct 30 01:36:17.702182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.702156    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"]
Oct 30 01:36:17.705363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.705331    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:17.705475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.705395    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:17.705475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:17.705427    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:18.228578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:18.228539    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:18.386874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:18.382669    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" event={"ID":"b78892cf-9422-4937-b888-0e7d5326fd13","Type":"ContainerStarted","Data":"60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"}
Oct 30 01:36:18.386874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:18.382748    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" event={"ID":"b78892cf-9422-4937-b888-0e7d5326fd13","Type":"ContainerStarted","Data":"c9c577580a900a2da7f1a78a4765879d6ec453f36c46e5a68bf4f8204dbc7d7b"}
Oct 30 01:36:18.484931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:18.484268    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused" start-of-body=
Oct 30 01:36:18.484931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:18.484330    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused"
Oct 30 01:36:19.328712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.328676    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:19.337005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.336982    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:19.392033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.392004    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:19.393666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.393644    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:36:19.415757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.415703    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podStartSLOduration=3.415686879 podStartE2EDuration="3.415686879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:36:18.463252151 +0000 UTC m=+3286.492562385" watchObservedRunningTime="2026-10-30 01:36:19.415686879 +0000 UTC m=+3287.444997110"
Oct 30 01:36:19.486877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.484986    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused" start-of-body=
Oct 30 01:36:19.486877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:19.485043    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused"
Oct 30 01:36:20.204526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:20.204478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:20.483827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:20.483795    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused" start-of-body=
Oct 30 01:36:20.484256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:20.483881    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.254:15021/healthz/ready\": dial tcp 10.134.0.254:15021: connect: connection refused"
Oct 30 01:36:20.912161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:20.912067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:21.038010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:21.037975    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:21.489323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:21.489287    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:36:21.489801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:21.489618    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:36:21.490408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:21.490392    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:36:22.204253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:22.204131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:23.764978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:23.764949    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:23.887122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:23.887083    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:24.232926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:24.223124    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:24.947805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:24.947747    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:25.715813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:25.715774    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:26.123581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:26.123545    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:27.016204 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:36:27.016165    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:36:27.059377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:27.059341    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:27.528179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:27.528130    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:27.905604 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:27.905521    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:28.095923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:28.086515    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:28.285876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:28.282288    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:28.405751 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:28.405696    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:28.684368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:28.684270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:29.420881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.417428    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"]
Oct 30 01:36:29.453568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.453404    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"]
Oct 30 01:36:29.453796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.453536    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.475278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.475223    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:29.502278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.501773    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5457f2-openshift-default-dockercfg-dlvjl\""
Oct 30 01:36:29.593490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.593438    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.593920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.593889    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594046    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594093    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5pvg\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594177    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594369    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:29.594480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594436    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594607    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.594705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.594689    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.702933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.699465    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.702933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.699578    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.702933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.699629    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.702933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.699663    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pvg\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.702933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.699698    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.702933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.702582    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.712023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.705055    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.715717    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.716000    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.716688    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.716990    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.717382    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.727713    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.727886    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.765880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.759882    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.772322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.772290    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.772962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.772936    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5pvg\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.790541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.790435    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo\") pod \"e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") " pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.794688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.794437    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:29.827413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:29.827351    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:30.131661 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:36:30.131631    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2687b657_b370_4dc5_bf86_516d3111160f.slice/crio-4c4d006c15e4c479017abe7803c9fb53779b1c75d8ba78e16b9730813008df06 WatchSource:0}: Error finding container 4c4d006c15e4c479017abe7803c9fb53779b1c75d8ba78e16b9730813008df06: Status 404 returned error can't find the container with id 4c4d006c15e4c479017abe7803c9fb53779b1c75d8ba78e16b9730813008df06
Oct 30 01:36:30.131914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.131892    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"]
Oct 30 01:36:30.135385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.135348    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:30.135527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.135433    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:30.135527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.135476    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:36:30.565164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.565131    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" event={"ID":"2687b657-b370-4dc5-bf86-516d3111160f","Type":"ContainerStarted","Data":"3d82c9a666a03b4e15bfe3badae79b02f052de1289684c7ccdeac8775ddb08da"}
Oct 30 01:36:30.565164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.565191    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" event={"ID":"2687b657-b370-4dc5-bf86-516d3111160f","Type":"ContainerStarted","Data":"4c4d006c15e4c479017abe7803c9fb53779b1c75d8ba78e16b9730813008df06"}
Oct 30 01:36:30.621324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.621260    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podStartSLOduration=1.621244079 podStartE2EDuration="1.621244079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:36:30.618225282 +0000 UTC m=+3298.647535515" watchObservedRunningTime="2026-10-30 01:36:30.621244079 +0000 UTC m=+3298.650554309"
Oct 30 01:36:30.806794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:30.800707    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:31.754343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:31.754302    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:31.864873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:31.864279    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:32.070103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:32.070023    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:32.182482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:32.182449    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:32.582725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:32.582693    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:32.583826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:32.583803    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:36:32.988196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:32.988144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:33.120343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:33.120308    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:33.758018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:33.757982    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:34.108765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:34.108687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:34.258960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:34.258926    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:35.757478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:35.757438    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:35.878491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:35.878457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:35.932204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:35.932171    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"]
Oct 30 01:36:35.937016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:35.936993    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:35.963974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:35.963942    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"]
Oct 30 01:36:36.021940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.008290    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5457f2-dockercfg-vnwcc\""
Oct 30 01:36:36.021940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.008422    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5457f2\""
Oct 30 01:36:36.060869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.035405    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqfh7\" (UniqueName: \"kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7\") pod \"payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.061209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.061175    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.333928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.326307    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqfh7\" (UniqueName: \"kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7\") pod \"payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.333928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.326384    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.333928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.327621    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.381912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.380413    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqfh7\" (UniqueName: \"kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7\") pod \"payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.560756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.560715    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:36.883026 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:36:36.882996    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc340ad11_f797_4128_abea_4ee3c7181708.slice/crio-483c26b5cb8c9c20cb9d570207dbe15d0b532a04848e33e46907fde8e1863d6b WatchSource:0}: Error finding container 483c26b5cb8c9c20cb9d570207dbe15d0b532a04848e33e46907fde8e1863d6b: Status 404 returned error can't find the container with id 483c26b5cb8c9c20cb9d570207dbe15d0b532a04848e33e46907fde8e1863d6b
Oct 30 01:36:36.884407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:36.884380    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"]
Oct 30 01:36:37.698503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:37.698471    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:37.721188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:37.721153    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" event={"ID":"c340ad11-f797-4128-abea-4ee3c7181708","Type":"ContainerStarted","Data":"d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f"}
Oct 30 01:36:37.721364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:37.721203    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" event={"ID":"c340ad11-f797-4128-abea-4ee3c7181708","Type":"ContainerStarted","Data":"483c26b5cb8c9c20cb9d570207dbe15d0b532a04848e33e46907fde8e1863d6b"}
Oct 30 01:36:37.721456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:37.721379    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:37.798550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:37.798480    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" podStartSLOduration=2.7984578559999997 podStartE2EDuration="2.798457856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:36:37.766489754 +0000 UTC m=+3305.795799986" watchObservedRunningTime="2026-10-30 01:36:37.798457856 +0000 UTC m=+3305.827768089"
Oct 30 01:36:39.420889 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:39.420855    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:41.150758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:41.150706    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:42.921289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:42.921250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:44.675527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:44.675488    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:46.517162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:46.517131    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:48.504805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:48.504758    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:48.755164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:48.755075    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:36:50.252229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:50.252190    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:51.215740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:51.215704    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:51.331338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:51.331292    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:52.005388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:52.005360    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:53.908701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:53.908660    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:55.618518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:55.618484    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:57.015672 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:36:57.015631    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:36:57.401519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:57.401434    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:36:59.151332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:36:59.151302    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:00.918717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:00.918682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:02.608616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:02.608568    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:04.289733 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:04.289682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:06.025785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:06.025748    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:07.725126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.725079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:07.947372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.947336    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"]
Oct 30 01:37:07.953786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.953746    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969334    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969466    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969533    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969558    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969589    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969627    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969657    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn7nh\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969766    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.969867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.969837    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:07.977684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:07.977606    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"]
Oct 30 01:37:08.017694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.017031    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bd287f-openshift-default-dockercfg-cn7cx\""
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074490    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074640    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074776    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074872    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074896    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074926    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.074964    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.075004    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn7nh\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.075313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.075124    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.076826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.076492    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.076826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.076526    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.076826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.076658    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.076826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.076790    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.077211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.077169    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.080641 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.080421    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.083359 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.083329    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.101890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.101808    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.102445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.102413    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn7nh\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh\") pod \"e2e-inf-a-bd287f-openshift-default-575f554785-stkj6\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") " pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.275298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.275256    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:08.624854 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.624801    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"]
Oct 30 01:37:08.625786 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:37:08.625757    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59836d05_c574_436c_9348_2d05f1faf029.slice/crio-0950147f49d3ca2ce8e1da4842c1076c8bf3b12a4ec7427c47aa5c1957bbba0f WatchSource:0}: Error finding container 0950147f49d3ca2ce8e1da4842c1076c8bf3b12a4ec7427c47aa5c1957bbba0f: Status 404 returned error can't find the container with id 0950147f49d3ca2ce8e1da4842c1076c8bf3b12a4ec7427c47aa5c1957bbba0f
Oct 30 01:37:08.630010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.629968    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:37:08.630157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.630069    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:37:08.630157 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:08.630116    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:37:09.481220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:09.481182    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" event={"ID":"59836d05-c574-436c-9348-2d05f1faf029","Type":"ContainerStarted","Data":"513ff0ccb8237c793c5e76e39edf5b2c92500a5f0067049603ca3afaca935db2"}
Oct 30 01:37:09.481685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:09.481238    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" event={"ID":"59836d05-c574-436c-9348-2d05f1faf029","Type":"ContainerStarted","Data":"0950147f49d3ca2ce8e1da4842c1076c8bf3b12a4ec7427c47aa5c1957bbba0f"}
Oct 30 01:37:09.525408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:09.525375    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:09.615865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:09.615746    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podStartSLOduration=2.615718579 podStartE2EDuration="2.615718579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:37:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:37:09.565249261 +0000 UTC m=+3337.594559518" watchObservedRunningTime="2026-10-30 01:37:09.615718579 +0000 UTC m=+3337.645028818"
Oct 30 01:37:10.275873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:10.275820    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:10.277865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:10.277823    4577 patch_prober.go:29] interesting pod/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused" start-of-body=
Oct 30 01:37:10.278013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:10.277910    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused"
Oct 30 01:37:11.275760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:11.275722    4577 patch_prober.go:29] interesting pod/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused" start-of-body=
Oct 30 01:37:11.276247 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:11.275789    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused"
Oct 30 01:37:11.333112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:11.333080    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:12.276447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:12.276408    4577 patch_prober.go:29] interesting pod/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused" start-of-body=
Oct 30 01:37:12.276867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:12.276472    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused"
Oct 30 01:37:13.204885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:13.204812    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:13.278696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:13.278247    4577 patch_prober.go:29] interesting pod/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused" start-of-body=
Oct 30 01:37:13.278696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:13.278323    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused"
Oct 30 01:37:14.276273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:14.276232    4577 patch_prober.go:29] interesting pod/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused" start-of-body=
Oct 30 01:37:14.276501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:14.276287    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.3:15021/healthz/ready\": dial tcp 10.134.1.3:15021: connect: connection refused"
Oct 30 01:37:14.888760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:14.888720    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:15.301530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:15.301472    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:15.302362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:15.302340    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:15.303695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:15.303672    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:37:16.635580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:16.635543    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:18.373864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:18.373813    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:18.489562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:18.489527    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:18.637289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:18.637196    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:20.306137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:20.306096    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:22.061223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:22.061169    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:23.816077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:23.816033    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:23.954132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:23.954095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:25.075695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:25.075648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:25.542836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:25.542803    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:27.015515 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:37:27.015477    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:37:27.286220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:27.286133    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:29.027136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:29.027095    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:30.776115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:30.776074    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:30.906141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:30.906043    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:31.092752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:31.092653    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:32.618824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:32.618733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:34.353508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:34.353471    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:34.467150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:34.467118    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:34.588609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:34.588574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:36.235266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:36.235236    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:38.022090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:38.022055    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:39.828452 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:39.828419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:41.534076 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:41.534032    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:43.294939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:43.294839    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:44.996426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:44.996386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:46.819255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:46.819219    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:48.570101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:48.570063    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:50.313108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:50.313049    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:51.178754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.178718    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"]
Oct 30 01:37:51.178959 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.178928    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy" containerID="cri-o://89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426" gracePeriod=30
Oct 30 01:37:51.193813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.193770    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"]
Oct 30 01:37:51.194153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.194107    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy" containerID="cri-o://b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd" gracePeriod=30
Oct 30 01:37:51.201291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.201255    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"]
Oct 30 01:37:51.201523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.201498    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy" containerID="cri-o://97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8" gracePeriod=30
Oct 30 01:37:51.997888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:51.997810    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:53.796556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:53.796519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:55.617729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:55.617693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:56.535236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.535203    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:37:56.583104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.583073    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:37:56.635415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.635389    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:37:56.677410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.677377    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.677687    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.677774    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.677878    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.677956    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.678053    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.678080    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.678111    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.678331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.678209    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.678819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.678375    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.679598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.679143    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:37:56.679598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.679301    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.679874    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.679982    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680019    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680055    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs\") pod \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\" (UID: \"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680095    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680130    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680159    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680216    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680304    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680349    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.680517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.680428    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.682296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.682274    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.682640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.682424    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.682640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.682448    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba96a3d3-146e-4de3-b926-57f2a456224c-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.682640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.682462    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.683105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.682919    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.683500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.683475    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:37:56.683712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.683694    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.683888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.683871    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.684109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.684093    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.686506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.686479    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.686736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.686711    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.686953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.686927    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.689642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.689618    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:37:56.689755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.689645    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:37:56.691388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.691362    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:37:56.693891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.693835    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.693999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.693908    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:37:56.700082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.700058    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.700678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.700642    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf" pod "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" (UID: "8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25"). InnerVolumeSpecName "kube-api-access-dmmhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:37:56.783862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.783810    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.784057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.783986    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.784057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784026    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.784190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784057    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.784190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784097    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.784190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784142    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.784369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784269    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert\") pod \"3f25f85b-9c1a-4783-ba88-796c04f572ee\" (UID: \"3f25f85b-9c1a-4783-ba88-796c04f572ee\") "
Oct 30 01:37:56.784369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784305    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.784369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784332    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token\") pod \"ba96a3d3-146e-4de3-b926-57f2a456224c\" (UID: \"ba96a3d3-146e-4de3-b926-57f2a456224c\") "
Oct 30 01:37:56.784746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.784699    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785132    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785532    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785558    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785571    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785584    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmmhf\" (UniqueName: \"kubernetes.io/projected/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-kube-api-access-dmmhf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785600    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785613    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785626    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25f85b-9c1a-4783-ba88-796c04f572ee-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785639    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785651    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785662    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785675    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785687    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785698    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785710    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785722    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785735    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25f85b-9c1a-4783-ba88-796c04f572ee-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785747    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.785784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.785761    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.787173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.787103    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:37:56.787284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.787192    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:37:56.787284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.787240    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:37:56.787398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.787369    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q" pod "3f25f85b-9c1a-4783-ba88-796c04f572ee" (UID: "3f25f85b-9c1a-4783-ba88-796c04f572ee"). InnerVolumeSpecName "kube-api-access-jhd2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:37:56.788067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.788043    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj" pod "ba96a3d3-146e-4de3-b926-57f2a456224c" (UID: "ba96a3d3-146e-4de3-b926-57f2a456224c"). InnerVolumeSpecName "kube-api-access-7qslj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:37:56.887642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.887605    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.887642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.887643    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba96a3d3-146e-4de3-b926-57f2a456224c-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.887952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.887657    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.887952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.887671    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qslj\" (UniqueName: \"kubernetes.io/projected/ba96a3d3-146e-4de3-b926-57f2a456224c-kube-api-access-7qslj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:56.887952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:56.887685    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhd2q\" (UniqueName: \"kubernetes.io/projected/3f25f85b-9c1a-4783-ba88-796c04f572ee-kube-api-access-jhd2q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:37:57.026743 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:37:57.026701    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:37:57.144315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.144228    4577 generic.go:356] "Generic (PLEG): container finished" podID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerID="89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426" exitCode=0
Oct 30 01:37:57.144315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.144296    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"
Oct 30 01:37:57.144574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.144311    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" event={"ID":"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25","Type":"ContainerDied","Data":"89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426"}
Oct 30 01:37:57.144574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.144378    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf" event={"ID":"8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25","Type":"ContainerDied","Data":"52e66468ef226ce58439f0604082a14602ae41368d1efd06f1f5b50ef9022ef1"}
Oct 30 01:37:57.144574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.144410    4577 scope.go:122] "RemoveContainer" containerID="89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426"
Oct 30 01:37:57.146140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.146102    4577 generic.go:356] "Generic (PLEG): container finished" podID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerID="b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd" exitCode=0
Oct 30 01:37:57.146300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.146212    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" event={"ID":"ba96a3d3-146e-4de3-b926-57f2a456224c","Type":"ContainerDied","Data":"b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd"}
Oct 30 01:37:57.146300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.146293    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9" event={"ID":"ba96a3d3-146e-4de3-b926-57f2a456224c","Type":"ContainerDied","Data":"64f1a05acc4f436aeed6f9e203f15ffea2b0e6444647562006f10a1eb3b1851d"}
Oct 30 01:37:57.146434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.146315    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"
Oct 30 01:37:57.148074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.148050    4577 generic.go:356] "Generic (PLEG): container finished" podID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerID="97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8" exitCode=0
Oct 30 01:37:57.148222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.148118    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" event={"ID":"3f25f85b-9c1a-4783-ba88-796c04f572ee","Type":"ContainerDied","Data":"97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8"}
Oct 30 01:37:57.148222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.148154    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2" event={"ID":"3f25f85b-9c1a-4783-ba88-796c04f572ee","Type":"ContainerDied","Data":"971c41ec97e3eeec7915f9d2e561ffab8e5d0a20f72036eb9d72579b3d96569e"}
Oct 30 01:37:57.148222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.148167    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"
Oct 30 01:37:57.164777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.164751    4577 scope.go:122] "RemoveContainer" containerID="89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426"
Oct 30 01:37:57.165336 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:37:57.165293    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426\": container with ID starting with 89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426 not found: ID does not exist" containerID="89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426"
Oct 30 01:37:57.165423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.165350    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426"} err="failed to get container status \"89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426\": rpc error: code = NotFound desc = could not find container \"89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426\": container with ID starting with 89b43314a63561172aa1be2a269b3e434fa5f0726d7d5bb97da82d3a1a70f426 not found: ID does not exist"
Oct 30 01:37:57.165423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.165376    4577 scope.go:122] "RemoveContainer" containerID="b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd"
Oct 30 01:37:57.187358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.187332    4577 scope.go:122] "RemoveContainer" containerID="b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd"
Oct 30 01:37:57.187803 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:37:57.187777    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd\": container with ID starting with b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd not found: ID does not exist" containerID="b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd"
Oct 30 01:37:57.187933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.187814    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd"} err="failed to get container status \"b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd\": rpc error: code = NotFound desc = could not find container \"b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd\": container with ID starting with b87981f6c81300c7725b5167f1090522a9c576b2f4625d700c9c041928b739bd not found: ID does not exist"
Oct 30 01:37:57.187933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.187839    4577 scope.go:122] "RemoveContainer" containerID="97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8"
Oct 30 01:37:57.201974 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.201949    4577 scope.go:122] "RemoveContainer" containerID="97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8"
Oct 30 01:37:57.202332 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:37:57.202292    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8\": container with ID starting with 97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8 not found: ID does not exist" containerID="97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8"
Oct 30 01:37:57.202480 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.202333    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8"} err="failed to get container status \"97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8\": rpc error: code = NotFound desc = could not find container \"97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8\": container with ID starting with 97e37353aa9e90bc5be92d2f9f08ff7d599ab4df3d9c136f142a7602466988d8 not found: ID does not exist"
Oct 30 01:37:57.210078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.210052    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"]
Oct 30 01:37:57.224046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.224016    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-rpgs9"]
Oct 30 01:37:57.256901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.256868    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"]
Oct 30 01:37:57.264198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.264167    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-9r8sf"]
Oct 30 01:37:57.303251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.303216    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"]
Oct 30 01:37:57.310918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.310862    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-bdcd2"]
Oct 30 01:37:57.384079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:57.384040    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:37:58.116956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:58.116919    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" path="/var/lib/kubelet/pods/3f25f85b-9c1a-4783-ba88-796c04f572ee/volumes"
Oct 30 01:37:58.117796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:58.117769    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" path="/var/lib/kubelet/pods/8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25/volumes"
Oct 30 01:37:58.118495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:58.118471    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" path="/var/lib/kubelet/pods/ba96a3d3-146e-4de3-b926-57f2a456224c/volumes"
Oct 30 01:37:59.358478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:37:59.358440    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:01.244502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:01.244468    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:03.111798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:03.111762    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:04.963874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:04.952317    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:06.693456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:06.693425    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:08.514045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:08.514003    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:10.228712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:10.228679    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:11.044422 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:38:11.044385    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78892cf_9422_4937_b888_0e7d5326fd13.slice/crio-60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78892cf_9422_4937_b888_0e7d5326fd13.slice/crio-conmon-60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37.scope\": RecentStats: unable to find data in memory cache], [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@46-49460-0.service\": RecentStats: unable to find data in memory cache]"
Oct 30 01:38:11.379988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.379950    4577 generic.go:356] "Generic (PLEG): container finished" podID="b78892cf-9422-4937-b888-0e7d5326fd13" containerID="60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37" exitCode=0
Oct 30 01:38:11.380502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.380025    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" event={"ID":"b78892cf-9422-4937-b888-0e7d5326fd13","Type":"ContainerDied","Data":"60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"}
Oct 30 01:38:11.380719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.380599    4577 scope.go:122] "RemoveContainer" containerID="60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"
Oct 30 01:38:11.382244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.382223    4577 generic.go:356] "Generic (PLEG): container finished" podID="2687b657-b370-4dc5-bf86-516d3111160f" containerID="3d82c9a666a03b4e15bfe3badae79b02f052de1289684c7ccdeac8775ddb08da" exitCode=0
Oct 30 01:38:11.382362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.382295    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" event={"ID":"2687b657-b370-4dc5-bf86-516d3111160f","Type":"ContainerDied","Data":"3d82c9a666a03b4e15bfe3badae79b02f052de1289684c7ccdeac8775ddb08da"}
Oct 30 01:38:11.382774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.382758    4577 scope.go:122] "RemoveContainer" containerID="3d82c9a666a03b4e15bfe3badae79b02f052de1289684c7ccdeac8775ddb08da"
Oct 30 01:38:11.383149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.383120    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:11.383208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.383193    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:11.383249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.383236    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:11.384904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.384875    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:11.385024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.384944    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:11.385024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:11.384976    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:12.021628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:12.021589    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:12.419952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:12.419872    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" event={"ID":"b78892cf-9422-4937-b888-0e7d5326fd13","Type":"ContainerStarted","Data":"b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833"}
Oct 30 01:38:12.422019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:12.421990    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" event={"ID":"2687b657-b370-4dc5-bf86-516d3111160f","Type":"ContainerStarted","Data":"59d08c1c96f2f924316aca4e4286cb0822a4c5b81a563253240a7a2b392b1303"}
Oct 30 01:38:12.806871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:12.800744    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:38:12.806871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:12.800921    4577 patch_prober.go:29] interesting pod/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused" start-of-body=
Oct 30 01:38:12.806871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:12.800981    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused"
Oct 30 01:38:13.329377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:13.329343    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:38:13.329630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:13.329611    4577 patch_prober.go:29] interesting pod/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused" start-of-body=
Oct 30 01:38:13.329682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:13.329654    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused"
Oct 30 01:38:13.804872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:13.803374    4577 patch_prober.go:29] interesting pod/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused" start-of-body=
Oct 30 01:38:13.804872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:13.803453    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused"
Oct 30 01:38:13.830378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:13.830347    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:14.329562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:14.329524    4577 patch_prober.go:29] interesting pod/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused" start-of-body=
Oct 30 01:38:14.329767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:14.329579    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused"
Oct 30 01:38:14.805912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:14.805585    4577 patch_prober.go:29] interesting pod/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused" start-of-body=
Oct 30 01:38:14.805912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:14.805711    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused"
Oct 30 01:38:15.329264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:15.329224    4577 patch_prober.go:29] interesting pod/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused" start-of-body=
Oct 30 01:38:15.329470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:15.329289    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused"
Oct 30 01:38:15.596933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:15.589578    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:15.800902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:15.800149    4577 patch_prober.go:29] interesting pod/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused" start-of-body=
Oct 30 01:38:15.800902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:15.800213    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused"
Oct 30 01:38:16.328928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:16.328892    4577 patch_prober.go:29] interesting pod/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused" start-of-body=
Oct 30 01:38:16.329347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:16.328953    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused"
Oct 30 01:38:16.803872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:16.803079    4577 patch_prober.go:29] interesting pod/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused" start-of-body=
Oct 30 01:38:16.803872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:16.803140    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused"
Oct 30 01:38:17.329102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:17.329067    4577 patch_prober.go:29] interesting pod/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused" start-of-body=
Oct 30 01:38:17.329566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:17.329125    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.0:15021/healthz/ready\": dial tcp 10.134.1.0:15021: connect: connection refused"
Oct 30 01:38:17.369937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:17.369873    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:17.630783 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:17.630572    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:38:17.805061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:17.805020    4577 patch_prober.go:29] interesting pod/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused" start-of-body=
Oct 30 01:38:17.813868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:17.812993    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.1:15021/healthz/ready\": dial tcp 10.134.1.1:15021: connect: connection refused"
Oct 30 01:38:18.334743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:18.334706    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:38:18.335234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:18.335147    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:38:18.335920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:18.335900    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:38:18.802958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:18.802926    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:38:18.804117 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:18.804095    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:38:19.124667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:19.124568    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:20.898328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:20.898294    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:22.631581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:22.631544    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:23.802548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:23.802493    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:24.273694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.273662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:24.535556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.535474    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:24.743445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.743410    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"]
Oct 30 01:38:24.744633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744610    4577 cpu_manager.go:414] "containerMap: removing container" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy"
Oct 30 01:38:24.744633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744632    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy"
Oct 30 01:38:24.744920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744683    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy"
Oct 30 01:38:24.744920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744691    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy"
Oct 30 01:38:24.744920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744714    4577 cpu_manager.go:414] "containerMap: removing container" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy"
Oct 30 01:38:24.744920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744722    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy"
Oct 30 01:38:24.744920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744899    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba96a3d3-146e-4de3-b926-57f2a456224c" containerName="istio-proxy"
Oct 30 01:38:24.744920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744916    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fa83ebe-aaad-4b21-9c86-35b4b2eaaf25" containerName="istio-proxy"
Oct 30 01:38:24.745177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.744934    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f25f85b-9c1a-4783-ba88-796c04f572ee" containerName="istio-proxy"
Oct 30 01:38:24.748565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.748545    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.784242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.784208    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"]
Oct 30 01:38:24.789662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.789586    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1a4b17-openshift-default-dockercfg-5rz5g\""
Oct 30 01:38:24.801949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.801915    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908055    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908124    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908268    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908347    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908400    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tc89\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908431    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908479    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908543    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.909873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.908664    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:24.966988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:24.966391    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.056821    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.056913    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tc89\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.056943    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.056981    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057028    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057106    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057265    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057298    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057418    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057836    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.057918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.057889    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.058688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.058141    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.058688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.058433    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.058688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.058660    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.104915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.100162    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.116985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.113537    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.116985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.114811    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.142484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.142405    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tc89\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89\") pod \"e2e-inf-b-1a4b17-openshift-default-655494677-wq82k\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") " pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.341614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.341530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:25.385109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.385078    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:25.768407 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:38:25.768378    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e195d5_f3d3_4e96_a1bc_6c898c909c63.slice/crio-105962dce5a97169ff93429015be57f446c29f30300abdab8add89fbe639f1d1 WatchSource:0}: Error finding container 105962dce5a97169ff93429015be57f446c29f30300abdab8add89fbe639f1d1: Status 404 returned error can't find the container with id 105962dce5a97169ff93429015be57f446c29f30300abdab8add89fbe639f1d1
Oct 30 01:38:25.769169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.769142    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"]
Oct 30 01:38:25.771663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.771627    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:25.771772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.771711    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:25.771772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.771761    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:38:25.805311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:25.805273    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" event={"ID":"c2e195d5-f3d3-4e96-a1bc-6c898c909c63","Type":"ContainerStarted","Data":"105962dce5a97169ff93429015be57f446c29f30300abdab8add89fbe639f1d1"}
Oct 30 01:38:26.413777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:26.413737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:26.818498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:26.818462    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" event={"ID":"c2e195d5-f3d3-4e96-a1bc-6c898c909c63","Type":"ContainerStarted","Data":"63198f222a246ab9a87b5a8f87506b37178a0ad38e5db18046bc9595ffe5e68d"}
Oct 30 01:38:26.885813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:26.885761    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podStartSLOduration=2.885746325 podStartE2EDuration="2.885746325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:38:26.880656182 +0000 UTC m=+3414.909966415" watchObservedRunningTime="2026-10-30 01:38:26.885746325 +0000 UTC m=+3414.915056559"
Oct 30 01:38:27.109103 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:38:27.109004    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:38:27.386164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:27.386082    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:27.392967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:27.392935    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:27.829559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:27.829527    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:27.831231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:27.831204    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:38:28.233681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:28.233648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:29.294370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:29.294336    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:29.449458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:29.449426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:30.271469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:30.271439    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:31.050955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.050918    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:31.165860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.165817    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:31.211645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.211606    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"]
Oct 30 01:38:31.219087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.219059    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.230893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.230811    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"]
Oct 30 01:38:31.237734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.237708    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-1a4b17-dockercfg-722mp\""
Oct 30 01:38:31.237943 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.237708    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-1a4b17\""
Oct 30 01:38:31.314528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.299013    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxrmp\" (UniqueName: \"kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp\") pod \"payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.440626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.440588    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.634939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.609019    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.634939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.616739    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.634939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.621191    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxrmp\" (UniqueName: \"kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp\") pod \"payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.651031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.650999    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"]
Oct 30 01:38:31.695909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.678187    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:31.713922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.702758    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxrmp\" (UniqueName: \"kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp\") pod \"payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.717862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.717798    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"]
Oct 30 01:38:31.724891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.724816    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:31.725187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.725162    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skh4b\" (UniqueName: \"kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b\") pod \"payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:31.826529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.826495    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skh4b\" (UniqueName: \"kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b\") pod \"payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:31.826724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.826650    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:31.827402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.827383    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:31.845996 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.845968    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:31.852330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:31.852298    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skh4b\" (UniqueName: \"kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b\") pod \"payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:32.031080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.031047    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:32.205943 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:38:32.205902    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eccd8f4_0d45_4a44_841c_a2b03a4bda04.slice/crio-e612fbe3d45db81a20764f79eedadee8834b69c9c92cb87b8d752e73c571b7f6 WatchSource:0}: Error finding container e612fbe3d45db81a20764f79eedadee8834b69c9c92cb87b8d752e73c571b7f6: Status 404 returned error can't find the container with id e612fbe3d45db81a20764f79eedadee8834b69c9c92cb87b8d752e73c571b7f6
Oct 30 01:38:32.208418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.208390    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"]
Oct 30 01:38:32.354524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.354358    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"]
Oct 30 01:38:32.416528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.416494    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:32.893298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.893258    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" event={"ID":"5eccd8f4-0d45-4a44-841c-a2b03a4bda04","Type":"ContainerStarted","Data":"652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f"}
Oct 30 01:38:32.893545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.893336    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:32.893545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.893360    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" event={"ID":"5eccd8f4-0d45-4a44-841c-a2b03a4bda04","Type":"ContainerStarted","Data":"e612fbe3d45db81a20764f79eedadee8834b69c9c92cb87b8d752e73c571b7f6"}
Oct 30 01:38:32.895308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.895273    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9" event={"ID":"a1761c8f-2e49-4669-ba3f-67d431adfb99","Type":"ContainerStarted","Data":"a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa"}
Oct 30 01:38:32.895453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.895331    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9" event={"ID":"a1761c8f-2e49-4669-ba3f-67d431adfb99","Type":"ContainerStarted","Data":"5acca6e0e0640beb30dff1eb07073db6524757511acf2bc899007a624a9862a8"}
Oct 30 01:38:32.895453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.895365    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:32.939149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.939078    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" podStartSLOduration=1.9390597010000001 podStartE2EDuration="1.939059701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:38:32.937269437 +0000 UTC m=+3420.966579668" watchObservedRunningTime="2026-10-30 01:38:32.939059701 +0000 UTC m=+3420.968369933"
Oct 30 01:38:32.986083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:32.986026    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9" podStartSLOduration=1.9860087119999998 podStartE2EDuration="1.986008712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:38:32.981515568 +0000 UTC m=+3421.010825800" watchObservedRunningTime="2026-10-30 01:38:32.986008712 +0000 UTC m=+3421.015318974"
Oct 30 01:38:34.218829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:34.218797    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:36.121705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:36.121665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:37.970205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:37.970162    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:39.918837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:39.918791    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:40.194513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:40.194414    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:40.968882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:40.968800    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:41.710030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:41.709999    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:41.935282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:41.935193    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:43.801083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:43.801045    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:43.912230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:43.912199    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:38:43.912550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:43.912522    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:38:45.516190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:45.516139    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:47.323418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:47.323375    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:49.176903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:49.176826    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:50.831045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:50.831015    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:52.633069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:52.633033    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:54.334367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:54.334325    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:56.029685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:56.029648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:57.015789 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:38:57.015752    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:38:57.726923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:57.726880    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:38:59.511711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:38:59.511654    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:01.318267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:01.318220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:03.065016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:03.064979    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:04.844008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:04.843972    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:06.798730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:06.798698    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:08.509681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:08.509630    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:09.125569 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:09.125522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:10.295445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:10.295406    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:10.418092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:10.418061    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:12.333083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:12.333041    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:13.821048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:13.821019    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:14.018750 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:14.018718    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:15.820453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:15.820415    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:17.642503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:17.642426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:19.510036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:19.510000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:21.311377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:21.311330    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:23.133286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:23.133246    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:24.850912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:24.850878    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:25.473456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:25.473422    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:26.519736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:26.519703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:27.014596 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:39:27.014560    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:39:28.244173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:28.244136    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:29.921949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:29.921911    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:31.655143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:31.655102    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:31.866925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:31.866892    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:33.619128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:33.619093    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:35.313790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:35.313750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:37.125558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:37.125524    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:38.804656 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:38.804625    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:40.577926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:40.577881    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:42.418037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:42.418001    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:44.107663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:44.107624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:45.863262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:45.863231    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:47.699695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:47.699666    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:49.404891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:49.404827    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:51.088033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:51.087993    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:52.869120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:52.869087    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:54.625962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:54.625927    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:56.343907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:56.343830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:39:57.427871 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:39:57.425522    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:39:58.163875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:39:58.150029    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:00.092209 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:00.092156    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:01.832757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:01.832698    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:03.726948 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:03.726916    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:05.504341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:05.504308    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:07.190394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:07.190356    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:08.929860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:08.929809    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:10.700972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:10.700942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:12.428449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:12.428411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:14.212783 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:14.212747    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:16.049059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:16.049020    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:17.816635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:17.816597    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:19.580020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:19.579989    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:20.330368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:20.330334    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:20.446918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:20.446883    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:21.561826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:21.561789    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:22.187739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:22.187701    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:23.313333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:23.313289    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:25.005746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:25.005715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:25.585145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:25.585107    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:26.716956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:26.716922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:27.016084 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:40:27.016047    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:40:28.403333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:28.403287    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:30.328175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:30.328148    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:32.086625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:32.086543    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:33.881603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:33.881568    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:35.741204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:35.741165    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:37.494890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:37.483053    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:39.314459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:39.314417    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:41.008057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:41.008020    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:42.496417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:42.496379    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:42.802919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:42.802760    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:44.714446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:44.714408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:44.988174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:44.987894    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:46.464503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:46.464467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:48.215699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.215662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:48.870252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.870221    4577 generic.go:356] "Generic (PLEG): container finished" podID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerID="63198f222a246ab9a87b5a8f87506b37178a0ad38e5db18046bc9595ffe5e68d" exitCode=0
Oct 30 01:40:48.870469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.870306    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" event={"ID":"c2e195d5-f3d3-4e96-a1bc-6c898c909c63","Type":"ContainerDied","Data":"63198f222a246ab9a87b5a8f87506b37178a0ad38e5db18046bc9595ffe5e68d"}
Oct 30 01:40:48.870815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.870795    4577 scope.go:122] "RemoveContainer" containerID="63198f222a246ab9a87b5a8f87506b37178a0ad38e5db18046bc9595ffe5e68d"
Oct 30 01:40:48.873088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.873061    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:40:48.873164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.873130    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:40:48.873218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:48.873166    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:40:49.890398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:49.890357    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" event={"ID":"c2e195d5-f3d3-4e96-a1bc-6c898c909c63","Type":"ContainerStarted","Data":"b3c830012cf12fdbcd4d48fd6a05ca03db40166c2b784aa0993aeb0827161e87"}
Oct 30 01:40:49.960531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:49.960418    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:50.385926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:50.385885    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:40:50.386137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:50.386113    4577 patch_prober.go:29] interesting pod/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused" start-of-body=
Oct 30 01:40:50.386189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:50.386160    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused"
Oct 30 01:40:51.386238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:51.386201    4577 patch_prober.go:29] interesting pod/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused" start-of-body=
Oct 30 01:40:51.386666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:51.386258    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused"
Oct 30 01:40:51.684369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:51.684271    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:52.386027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:52.385833    4577 patch_prober.go:29] interesting pod/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused" start-of-body=
Oct 30 01:40:52.386027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:52.385949    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused"
Oct 30 01:40:53.386485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:53.386444    4577 patch_prober.go:29] interesting pod/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused" start-of-body=
Oct 30 01:40:53.386957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:53.386522    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused"
Oct 30 01:40:53.431396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:53.431362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:54.385655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:54.385614    4577 patch_prober.go:29] interesting pod/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused" start-of-body=
Oct 30 01:40:54.385868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:54.385696    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.4:15021/healthz/ready\": dial tcp 10.134.1.4:15021: connect: connection refused"
Oct 30 01:40:55.113040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:55.112998    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:55.392417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:55.392329    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:40:55.392893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:55.392840    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:40:55.393617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:55.393597    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:40:56.851280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:56.851249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:40:57.016110 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:40:57.016072    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:40:58.805134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:40:58.805104    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:00.615227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:00.615191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:02.360221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:02.360177    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:04.231218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:04.231184    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:06.004064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:06.004010    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:07.742540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:07.742481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:09.528966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:09.528931    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:11.268234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:11.268197    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:13.002121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:13.002084    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:13.459112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:13.458984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:14.894466 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:14.894420    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:15.001669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.001633    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"]
Oct 30 01:41:15.015906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.015877    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.017286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.017258    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"]
Oct 30 01:41:15.047062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.046241    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:41:15.047062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.046316    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 01:41:15.120380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.120297    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.120594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.120487    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvk7g\" (UniqueName: \"kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.121328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.121298    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.223453 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.223356    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.223655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.223475    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.223655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.223512    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvk7g\" (UniqueName: \"kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.224339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.224317    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.226152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.226130    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.249056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.249019    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvk7g\" (UniqueName: \"kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g\") pod \"maas-api-revoke-sub-31383514-9t4zt\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.359560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.359527    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:15.677858 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:41:15.677794    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f618bbf_46e0_4a73_8f29_23c76f1928f0.slice/crio-69edf8a39a837bc51a4fca4724cbada666415c9563386fdc28e579b6d14865b8 WatchSource:0}: Error finding container 69edf8a39a837bc51a4fca4724cbada666415c9563386fdc28e579b6d14865b8: Status 404 returned error can't find the container with id 69edf8a39a837bc51a4fca4724cbada666415c9563386fdc28e579b6d14865b8
Oct 30 01:41:15.679916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:15.679886    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"]
Oct 30 01:41:16.210483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:16.210444    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt" event={"ID":"6f618bbf-46e0-4a73-8f29-23c76f1928f0","Type":"ContainerStarted","Data":"fcae50843c889879b2b6047a0d21a9601abc92aed898f4e20870396302ae045e"}
Oct 30 01:41:16.210919 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:16.210523    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt" event={"ID":"6f618bbf-46e0-4a73-8f29-23c76f1928f0","Type":"ContainerStarted","Data":"69edf8a39a837bc51a4fca4724cbada666415c9563386fdc28e579b6d14865b8"}
Oct 30 01:41:16.259491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:16.259433    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt" podStartSLOduration=2.259418334 podStartE2EDuration="2.259418334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:41:16.25421647 +0000 UTC m=+3584.283526701" watchObservedRunningTime="2026-10-30 01:41:16.259418334 +0000 UTC m=+3584.288728564"
Oct 30 01:41:16.716180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:16.716100    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:17.222366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:17.222330    4577 generic.go:356] "Generic (PLEG): container finished" podID="6f618bbf-46e0-4a73-8f29-23c76f1928f0" containerID="fcae50843c889879b2b6047a0d21a9601abc92aed898f4e20870396302ae045e" exitCode=0
Oct 30 01:41:17.222881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:17.222421    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt" event={"ID":"6f618bbf-46e0-4a73-8f29-23c76f1928f0","Type":"ContainerDied","Data":"fcae50843c889879b2b6047a0d21a9601abc92aed898f4e20870396302ae045e"}
Oct 30 01:41:18.565630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:18.564409    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:18.877816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:18.872273    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"]
Oct 30 01:41:18.877816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:18.872531    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy" containerID="cri-o://ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de" gracePeriod=30
Oct 30 01:41:19.002315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.002282    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:19.090526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.090493    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g\" (UniqueName: \"kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g\") pod \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") "
Oct 30 01:41:19.091059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.091032    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca\") pod \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") "
Oct 30 01:41:19.091619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.091593    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca\") pod \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\" (UID: \"6f618bbf-46e0-4a73-8f29-23c76f1928f0\") "
Oct 30 01:41:19.093119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.093095    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca" pod "6f618bbf-46e0-4a73-8f29-23c76f1928f0" (UID: "6f618bbf-46e0-4a73-8f29-23c76f1928f0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:41:19.093990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.093963    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g" pod "6f618bbf-46e0-4a73-8f29-23c76f1928f0" (UID: "6f618bbf-46e0-4a73-8f29-23c76f1928f0"). InnerVolumeSpecName "kube-api-access-jvk7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:41:19.094186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.094161    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca" pod "6f618bbf-46e0-4a73-8f29-23c76f1928f0" (UID: "6f618bbf-46e0-4a73-8f29-23c76f1928f0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:41:19.193707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.193621    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:19.193707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.193650    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6f618bbf-46e0-4a73-8f29-23c76f1928f0-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:19.193707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.193660    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvk7g\" (UniqueName: \"kubernetes.io/projected/6f618bbf-46e0-4a73-8f29-23c76f1928f0-kube-api-access-jvk7g\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:19.330309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.330276    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"
Oct 30 01:41:19.330526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.330378    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt" event={"ID":"6f618bbf-46e0-4a73-8f29-23c76f1928f0","Type":"ContainerDied","Data":"69edf8a39a837bc51a4fca4724cbada666415c9563386fdc28e579b6d14865b8"}
Oct 30 01:41:19.330526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:19.330456    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69edf8a39a837bc51a4fca4724cbada666415c9563386fdc28e579b6d14865b8"
Oct 30 01:41:20.295687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:20.295650    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:22.114058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:22.114023    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:23.163085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:23.163040    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:24.023582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.023546    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:24.267041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.267011    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:41:24.392495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.392462    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.392827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.392788    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:24.393410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.393388    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:41:24.393661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.393605    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.393978    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.394058    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.394182    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.394219    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.394296    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.394366    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.394470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.394456    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs\") pod \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\" (UID: \"9379b6ab-f35d-4d33-bca6-9c9073c290f9\") "
Oct 30 01:41:24.395336 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.395315    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.395416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.395343    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.395599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.395579    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:24.395812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.395793    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:24.397061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.397035    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:24.403083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.403045    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:24.403188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.403123    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:41:24.405621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.404182    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:41:24.409608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.406025    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq" pod "9379b6ab-f35d-4d33-bca6-9c9073c290f9" (UID: "9379b6ab-f35d-4d33-bca6-9c9073c290f9"). InnerVolumeSpecName "kube-api-access-55hrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:41:24.424040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.424008    4577 generic.go:356] "Generic (PLEG): container finished" podID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerID="ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de" exitCode=0
Oct 30 01:41:24.424205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.424084    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"
Oct 30 01:41:24.424205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.424078    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" event={"ID":"9379b6ab-f35d-4d33-bca6-9c9073c290f9","Type":"ContainerDied","Data":"ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de"}
Oct 30 01:41:24.424341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.424218    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2" event={"ID":"9379b6ab-f35d-4d33-bca6-9c9073c290f9","Type":"ContainerDied","Data":"47d92ff8920be8ff639ba2702bd6a172aa44b53289bdad7cd9aa4a609dd85dc5"}
Oct 30 01:41:24.424341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.424250    4577 scope.go:122] "RemoveContainer" containerID="ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de"
Oct 30 01:41:24.456688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.456664    4577 scope.go:122] "RemoveContainer" containerID="a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"
Oct 30 01:41:24.468906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.468881    4577 scope.go:122] "RemoveContainer" containerID="ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de"
Oct 30 01:41:24.469216 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:41:24.469194    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de\": container with ID starting with ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de not found: ID does not exist" containerID="ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de"
Oct 30 01:41:24.469288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.469225    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de"} err="failed to get container status \"ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de\": rpc error: code = NotFound desc = could not find container \"ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de\": container with ID starting with ad8972ac1435f89388e34c889f5b69232fd31e11435c45b3b6a5acb5080106de not found: ID does not exist"
Oct 30 01:41:24.469288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.469243    4577 scope.go:122] "RemoveContainer" containerID="a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"
Oct 30 01:41:24.469604 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:41:24.469580    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472\": container with ID starting with a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472 not found: ID does not exist" containerID="a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"
Oct 30 01:41:24.469675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.469612    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472"} err="failed to get container status \"a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472\": rpc error: code = NotFound desc = could not find container \"a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472\": container with ID starting with a97a1af1222e8f2497cf7d260d36f507d9c9360d35f8219795eeb0e59b5bf472 not found: ID does not exist"
Oct 30 01:41:24.496506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496414    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"]
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496563    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496581    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55hrq\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-kube-api-access-55hrq\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496591    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496601    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496609    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496616    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9379b6ab-f35d-4d33-bca6-9c9073c290f9-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.496694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.496624    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9379b6ab-f35d-4d33-bca6-9c9073c290f9-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:24.501284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:24.501256    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-0e1a98-openshift-default-7f9b786d5c-l2ks2"]
Oct 30 01:41:25.714961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:25.714929    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:25.999295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:25.999256    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:26.118200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:26.118160    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" path="/var/lib/kubelet/pods/9379b6ab-f35d-4d33-bca6-9c9073c290f9/volumes"
Oct 30 01:41:27.020437 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:41:27.020400    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:41:27.769034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:27.769000    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:29.681439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:29.681399    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:31.396978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:31.396944    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:33.254905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:33.254828    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:35.023404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:35.023371    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:36.817857 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:36.817822    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:37.220653 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:37.220552    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:38.699315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:38.699270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:40.265381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.265339    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:40.511610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.511569    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"]
Oct 30 01:41:40.512645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512615    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy"
Oct 30 01:41:40.512645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512643    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy"
Oct 30 01:41:40.512880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512697    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6f618bbf-46e0-4a73-8f29-23c76f1928f0" containerName="revoke-keys"
Oct 30 01:41:40.512880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512705    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f618bbf-46e0-4a73-8f29-23c76f1928f0" containerName="revoke-keys"
Oct 30 01:41:40.512999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512894    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy"
Oct 30 01:41:40.512999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512922    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy"
Oct 30 01:41:40.512999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.512932    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f618bbf-46e0-4a73-8f29-23c76f1928f0" containerName="revoke-keys"
Oct 30 01:41:40.517287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.517225    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.541147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.541111    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"]
Oct 30 01:41:40.542329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.542294    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 01:41:40.542633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.542611    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:41:40.600952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.600918    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.601131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.601094    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.601309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.601292    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swz5d\" (UniqueName: \"kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.689302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.689254    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:40.702515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.702485    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swz5d\" (UniqueName: \"kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.702689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.702602    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.702760 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.702745    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.703326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.703305    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.705312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.705289    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.727712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.727682    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swz5d\" (UniqueName: \"kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d\") pod \"maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:40.837352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:40.837254    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:41.197269 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:41:41.197236    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbc85a4_749c_4b50_be11_cf23de742321.slice/crio-27eda2112091a59263a9df80fd33e206ae85c7523d95c6c7f885d1341b75062d WatchSource:0}: Error finding container 27eda2112091a59263a9df80fd33e206ae85c7523d95c6c7f885d1341b75062d: Status 404 returned error can't find the container with id 27eda2112091a59263a9df80fd33e206ae85c7523d95c6c7f885d1341b75062d
Oct 30 01:41:41.200484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:41.200457    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"]
Oct 30 01:41:41.597403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:41.597372    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2" event={"ID":"6fbc85a4-749c-4b50-be11-cf23de742321","Type":"ContainerStarted","Data":"0ea3d5db2297480d74a9b8a168a9bc41b55653e97620b79e318e83acc1510c78"}
Oct 30 01:41:41.597892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:41.597427    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2" event={"ID":"6fbc85a4-749c-4b50-be11-cf23de742321","Type":"ContainerStarted","Data":"27eda2112091a59263a9df80fd33e206ae85c7523d95c6c7f885d1341b75062d"}
Oct 30 01:41:41.643362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:41.643315    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2" podStartSLOduration=1.643301004 podStartE2EDuration="1.643301004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:41:41.640973722 +0000 UTC m=+3609.670283952" watchObservedRunningTime="2026-10-30 01:41:41.643301004 +0000 UTC m=+3609.672611235"
Oct 30 01:41:42.609915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:42.609877    4577 generic.go:356] "Generic (PLEG): container finished" podID="6fbc85a4-749c-4b50-be11-cf23de742321" containerID="0ea3d5db2297480d74a9b8a168a9bc41b55653e97620b79e318e83acc1510c78" exitCode=0
Oct 30 01:41:42.610326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:42.609972    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2" event={"ID":"6fbc85a4-749c-4b50-be11-cf23de742321","Type":"ContainerDied","Data":"0ea3d5db2297480d74a9b8a168a9bc41b55653e97620b79e318e83acc1510c78"}
Oct 30 01:41:42.631041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:42.631012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:43.802689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.802664    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:43.963636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.963555    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca\") pod \"6fbc85a4-749c-4b50-be11-cf23de742321\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") "
Oct 30 01:41:43.963786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.963734    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca\") pod \"6fbc85a4-749c-4b50-be11-cf23de742321\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") "
Oct 30 01:41:43.963870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.963797    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d\" (UniqueName: \"kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d\") pod \"6fbc85a4-749c-4b50-be11-cf23de742321\" (UID: \"6fbc85a4-749c-4b50-be11-cf23de742321\") "
Oct 30 01:41:43.964234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.964170    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca" pod "6fbc85a4-749c-4b50-be11-cf23de742321" (UID: "6fbc85a4-749c-4b50-be11-cf23de742321"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:41:43.964575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.964554    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:43.966114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.966079    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca" pod "6fbc85a4-749c-4b50-be11-cf23de742321" (UID: "6fbc85a4-749c-4b50-be11-cf23de742321"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:41:43.966275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:43.966251    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d" pod "6fbc85a4-749c-4b50-be11-cf23de742321" (UID: "6fbc85a4-749c-4b50-be11-cf23de742321"). InnerVolumeSpecName "kube-api-access-swz5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:41:44.066161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:44.066128    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swz5d\" (UniqueName: \"kubernetes.io/projected/6fbc85a4-749c-4b50-be11-cf23de742321-kube-api-access-swz5d\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:44.066161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:44.066158    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6fbc85a4-749c-4b50-be11-cf23de742321-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:44.493294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:44.493241    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:44.634483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:44.634449    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"
Oct 30 01:41:44.634732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:44.634464    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2" event={"ID":"6fbc85a4-749c-4b50-be11-cf23de742321","Type":"ContainerDied","Data":"27eda2112091a59263a9df80fd33e206ae85c7523d95c6c7f885d1341b75062d"}
Oct 30 01:41:44.634937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:44.634917    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27eda2112091a59263a9df80fd33e206ae85c7523d95c6c7f885d1341b75062d"
Oct 30 01:41:46.291035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:46.291003    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:48.118664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:48.118629    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:49.895159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:49.895129    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:50.411533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:50.411496    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"]
Oct 30 01:41:50.411784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:50.411759    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" containerID="cri-o://b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833" gracePeriod=30
Oct 30 01:41:51.154278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:51.154243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:51.298704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:51.298670    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:51.458921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:51.458788    4577 patch_prober.go:29] interesting pod/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:41:51.458921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:51.458882    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:41:51.630550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:51.630518    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:53.418661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:53.418629    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:53.827217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:53.827183    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:55.536091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.535928    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:55.743879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.743855    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 01:41:55.782320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782288    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782397    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782502    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782568    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782632    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782633    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:55.782942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782710    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782836    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:55.782942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782877    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.782942 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782924    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.783167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.782954    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert\") pod \"e0856659-4315-48f2-be0f-def689ca8af3\" (UID: \"e0856659-4315-48f2-be0f-def689ca8af3\") "
Oct 30 01:41:55.783167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.783056    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:55.783436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.783412    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:41:55.783613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.783469    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:55.784235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.784208    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0856659-4315-48f2-be0f-def689ca8af3-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.784371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.784239    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.784371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.784253    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.784371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.784267    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.784371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.784280    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.785366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.785341    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:41:55.785498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.785479    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "kube-api-access-k44h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:41:55.785920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.785891    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:41:55.785982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.785912    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo" pod "e0856659-4315-48f2-be0f-def689ca8af3" (UID: "e0856659-4315-48f2-be0f-def689ca8af3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:41:55.828308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.828265    4577 generic.go:356] "Generic (PLEG): container finished" podID="e0856659-4315-48f2-be0f-def689ca8af3" containerID="b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833" exitCode=0
Oct 30 01:41:55.828861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.828403    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" event={"ID":"e0856659-4315-48f2-be0f-def689ca8af3","Type":"ContainerDied","Data":"b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833"}
Oct 30 01:41:55.828861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.828655    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh" event={"ID":"e0856659-4315-48f2-be0f-def689ca8af3","Type":"ContainerDied","Data":"8a8ba8618bb4e7505ef39257fc27bd5b336831cb6c74c554d713952d70e98ead"}
Oct 30 01:41:55.828861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.828494    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"
Oct 30 01:41:55.828861 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.828693    4577 scope.go:122] "RemoveContainer" containerID="b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833"
Oct 30 01:41:55.861625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.860705    4577 scope.go:122] "RemoveContainer" containerID="b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833"
Oct 30 01:41:55.862516 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:41:55.862477    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833\": container with ID starting with b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833 not found: ID does not exist" containerID="b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833"
Oct 30 01:41:55.862644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.862526    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833"} err="failed to get container status \"b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833\": rpc error: code = NotFound desc = could not find container \"b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833\": container with ID starting with b8d306ab8aa9e9010e20cda3f028f5ff897c4328344af85042c0a641e1340833 not found: ID does not exist"
Oct 30 01:41:55.885490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.885460    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0856659-4315-48f2-be0f-def689ca8af3-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.885490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.885491    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0856659-4315-48f2-be0f-def689ca8af3-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.885714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.885503    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.885714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.885514    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k44h2\" (UniqueName: \"kubernetes.io/projected/e0856659-4315-48f2-be0f-def689ca8af3-kube-api-access-k44h2\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:41:55.905784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.905753    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"]
Oct 30 01:41:55.912408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:55.912384    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-d5b2b8-openshift-default-fcbcf95f9-hkszh"]
Oct 30 01:41:56.114085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:56.114056    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0856659-4315-48f2-be0f-def689ca8af3" path="/var/lib/kubelet/pods/e0856659-4315-48f2-be0f-def689ca8af3/volumes"
Oct 30 01:41:57.014990 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:41:57.014959    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:41:57.292976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:57.292823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:41:59.120745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:41:59.120713    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:00.572906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:00.572834    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:01.107871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:01.103632    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:01.216374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:01.216341    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:01.279953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:01.279905    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"]
Oct 30 01:42:01.293977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:01.293945    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bd287f-s52f2"]
Oct 30 01:42:01.747545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:01.747508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:02.115179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:02.115102    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fbc85a4-749c-4b50-be11-cf23de742321" path="/var/lib/kubelet/pods/6fbc85a4-749c-4b50-be11-cf23de742321/volumes"
Oct 30 01:42:03.181868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:03.172536    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:03.926572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:03.926537    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:04.045014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:04.044981    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:04.660721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:04.660682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:04.798446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:04.798398    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:05.015347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:05.015304    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:06.043632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:06.042750    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" probeResult="failure" output="Get \"https://10.134.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:42:06.915053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:06.915016    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:08.924309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:08.924276    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:09.664873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:09.664827    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:09.777021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:09.776982    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:09.971753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:09.971663    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"]
Oct 30 01:42:09.972274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:09.971989    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy" containerID="cri-o://513ff0ccb8237c793c5e76e39edf5b2c92500a5f0067049603ca3afaca935db2" gracePeriod=30
Oct 30 01:42:10.015935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:10.015885    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:10.770124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:10.770088    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:12.766600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:12.766515    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:14.714463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:14.714426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:15.240924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.240804    4577 generic.go:356] "Generic (PLEG): container finished" podID="59836d05-c574-436c-9348-2d05f1faf029" containerID="513ff0ccb8237c793c5e76e39edf5b2c92500a5f0067049603ca3afaca935db2" exitCode=0
Oct 30 01:42:15.240924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.240884    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" event={"ID":"59836d05-c574-436c-9348-2d05f1faf029","Type":"ContainerDied","Data":"513ff0ccb8237c793c5e76e39edf5b2c92500a5f0067049603ca3afaca935db2"}
Oct 30 01:42:15.273989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.273964    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:42:15.411418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411384    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411428    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411463    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411491    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411537    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411617    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411808    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411859    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.411941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411911    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs\") pod \"59836d05-c574-436c-9348-2d05f1faf029\" (UID: \"59836d05-c574-436c-9348-2d05f1faf029\") "
Oct 30 01:42:15.412120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411950    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:42:15.412120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.411915    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:42:15.412246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.412112    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:42:15.412246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.412188    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:42:15.412246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.412233    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:42:15.413005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.412973    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59836d05-c574-436c-9348-2d05f1faf029-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.413123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.413008    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.413123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.413026    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.413123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.413039    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.413123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.413052    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.414169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.414139    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:42:15.414282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.414223    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:42:15.414358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.414341    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:42:15.414419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.414390    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh" pod "59836d05-c574-436c-9348-2d05f1faf029" (UID: "59836d05-c574-436c-9348-2d05f1faf029"). InnerVolumeSpecName "kube-api-access-jn7nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:42:15.514684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.514600    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.514684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.514635    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn7nh\" (UniqueName: \"kubernetes.io/projected/59836d05-c574-436c-9348-2d05f1faf029-kube-api-access-jn7nh\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.514684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.514646    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59836d05-c574-436c-9348-2d05f1faf029-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:15.514684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:15.514669    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59836d05-c574-436c-9348-2d05f1faf029-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:42:16.253867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:16.253822    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"
Oct 30 01:42:16.254274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:16.253840    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6" event={"ID":"59836d05-c574-436c-9348-2d05f1faf029","Type":"ContainerDied","Data":"0950147f49d3ca2ce8e1da4842c1076c8bf3b12a4ec7427c47aa5c1957bbba0f"}
Oct 30 01:42:16.254274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:16.253939    4577 scope.go:122] "RemoveContainer" containerID="513ff0ccb8237c793c5e76e39edf5b2c92500a5f0067049603ca3afaca935db2"
Oct 30 01:42:16.328527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:16.328468    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"]
Oct 30 01:42:16.342126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:16.342095    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bd287f-openshift-default-575f554785-stkj6"]
Oct 30 01:42:16.521609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:16.521562    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:18.136489 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:18.136443    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59836d05-c574-436c-9348-2d05f1faf029" path="/var/lib/kubelet/pods/59836d05-c574-436c-9348-2d05f1faf029/volumes"
Oct 30 01:42:18.310643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:18.310606    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:20.149756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:20.149719    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:20.616822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:20.616792    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:20.732164 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:20.732126    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:21.129583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:21.129548    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:21.280425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:21.280387    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:21.885233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:21.885185    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:23.595657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:23.595624    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:25.416624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:25.416581    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:25.840150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:25.840116    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:27.015865 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:42:27.015816    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:42:27.198002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:27.197972    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:28.933832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:28.933797    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:30.698715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:30.698674    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:32.497281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:32.497248    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:34.225296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:34.225265    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:35.976187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:35.976142    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:37.699865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:37.699821    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:39.598854 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:39.598799    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:41.486050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:41.486014    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:43.209630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:43.209584    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:44.993937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:44.993894    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:46.980470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:46.980436    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:48.705258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:48.705221    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:50.272210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:50.272178    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:50.834558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:50.834478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:50.976382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:50.976349    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:51.472187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:51.472154    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:52.938768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:52.938732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:54.744524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:54.744493    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:56.600496 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:56.600463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:57.016060 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:42:57.016022    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:42:58.373718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:58.373677    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:58.504806 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:58.504777    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:58.702949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:58.702836    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"]
Oct 30 01:42:58.703437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:58.703400    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" containerID="cri-o://7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a" gracePeriod=30
Oct 30 01:42:58.716805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:58.716747    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:42:59.924053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:59.924017    4577 patch_prober.go:29] interesting pod/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:42:59.924479 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:42:59.924086    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:43:00.447970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:00.447929    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:02.330313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:02.330254    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:04.016188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.016158    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:43:04.028553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.028520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:04.080067 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.079990    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.080441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.080421    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.084207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.080675    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.098401 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.084500    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.099008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.098778    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.099395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.099375    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.100123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.100103    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.100334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.100305    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.100491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.100475    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs\") pod \"434adec4-4a45-460c-a1c8-36cfbdd2092a\" (UID: \"434adec4-4a45-460c-a1c8-36cfbdd2092a\") "
Oct 30 01:43:04.101994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.101942    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:04.102328 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.102305    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:04.102461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.102334    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:43:04.103878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.103824    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:04.119091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.119050    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "kube-api-access-b6697". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:43:04.120562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.120518    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:04.132895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.132007    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:43:04.134442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.134399    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:43:04.143664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.143502    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy" pod "434adec4-4a45-460c-a1c8-36cfbdd2092a" (UID: "434adec4-4a45-460c-a1c8-36cfbdd2092a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:04.203323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203275    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203305    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434adec4-4a45-460c-a1c8-36cfbdd2092a-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203320    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203330    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6697\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-kube-api-access-b6697\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203341    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203351    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203360    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203368    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434adec4-4a45-460c-a1c8-36cfbdd2092a-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.203628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.203376    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434adec4-4a45-460c-a1c8-36cfbdd2092a-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:04.839419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.839367    4577 generic.go:356] "Generic (PLEG): container finished" podID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerID="7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a" exitCode=0
Oct 30 01:43:04.839990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.839970    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" event={"ID":"434adec4-4a45-460c-a1c8-36cfbdd2092a","Type":"ContainerDied","Data":"7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a"}
Oct 30 01:43:04.840181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.840158    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22" event={"ID":"434adec4-4a45-460c-a1c8-36cfbdd2092a","Type":"ContainerDied","Data":"6b54f3804d94a07fd3cf99439d3c20991c2c6c888ac1a71b6e2bbb65c2e371bf"}
Oct 30 01:43:04.840308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.840296    4577 scope.go:122] "RemoveContainer" containerID="7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a"
Oct 30 01:43:04.840647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:04.840613    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"
Oct 30 01:43:05.038719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.038683    4577 scope.go:122] "RemoveContainer" containerID="f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"
Oct 30 01:43:05.071269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.062406    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"]
Oct 30 01:43:05.071269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.069155    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-415c6b-openshift-default-79df74bbc7-fbd22"]
Oct 30 01:43:05.077238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.076458    4577 scope.go:122] "RemoveContainer" containerID="7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a"
Oct 30 01:43:05.077238 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:05.077055    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a\": container with ID starting with 7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a not found: ID does not exist" containerID="7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a"
Oct 30 01:43:05.077238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.077097    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a"} err="failed to get container status \"7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a\": rpc error: code = NotFound desc = could not find container \"7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a\": container with ID starting with 7307c2355a9568e96d7e16fdb0d347939b1b9d4c85dc76376d85668c2611255a not found: ID does not exist"
Oct 30 01:43:05.077238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.077124    4577 scope.go:122] "RemoveContainer" containerID="f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"
Oct 30 01:43:05.077694 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:05.077667    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c\": container with ID starting with f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c not found: ID does not exist" containerID="f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"
Oct 30 01:43:05.077786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.077697    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c"} err="failed to get container status \"f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c\": rpc error: code = NotFound desc = could not find container \"f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c\": container with ID starting with f897dfbe5de416cbb2ab42fb34f586208db81ebef7fd235b705d59133d0c438c not found: ID does not exist"
Oct 30 01:43:05.924510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:05.924478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:06.114502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:06.114466    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" path="/var/lib/kubelet/pods/434adec4-4a45-460c-a1c8-36cfbdd2092a/volumes"
Oct 30 01:43:07.680066 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:07.680028    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:09.396294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:09.396263    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:09.746693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:09.746654    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"]
Oct 30 01:43:09.898794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:09.898738    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:10.021897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:10.021737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:11.162116 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:11.162047    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:12.690701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:12.690648    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:13.302876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:13.300477    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:13.414191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:13.414153    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:13.942640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:13.942606    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:14.747439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:14.747396    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing" containerID="cri-o://12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1" gracePeriod=25
Oct 30 01:43:14.978440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:14.978396    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:43:15.198816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:15.198728    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:17.020910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:17.020878    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:18.727112 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:18.727075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:20.224979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:20.224948    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:20.334986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:20.334936    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:20.536476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:20.536433    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:20.552877 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:20.552828    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"]
Oct 30 01:43:20.553495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:20.553104    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" containerID="cri-o://c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308" gracePeriod=30
Oct 30 01:43:20.650887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:20.649766    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:21.591654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:21.591620    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:21.713518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:21.713491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:21.833619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:21.833590    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:22.308463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:22.308431    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:22.494577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:22.494540    4577 patch_prober.go:29] interesting pod/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:43:22.494771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:22.494605    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:43:24.014625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:24.014574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:24.977277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:24.977239    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:43:25.845745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.845700    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:25.856296 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.856254    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:43:25.940250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940215    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.940463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940272    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.940463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940456    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.940624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940603    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:25.940699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940637    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.940699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940691    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.940837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940757    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:25.940837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940808    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.940999 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.940926    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.941064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.941019    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.941130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.941089    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy\") pod \"7603959c-91b2-462c-81e7-98e0799baebf\" (UID: \"7603959c-91b2-462c-81e7-98e0799baebf\") "
Oct 30 01:43:25.941229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.941201    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:43:25.941297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.941238    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:25.941416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.941390    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:25.942410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.942155    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:25.942410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.942294    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:25.942410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.942314    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:25.942410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.942329    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:25.942410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.942346    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7603959c-91b2-462c-81e7-98e0799baebf-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:25.943129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.943103    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "kube-api-access-xvpf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:43:25.944226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.944200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:43:25.944392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.944360    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:43:25.945393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.945357    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy" pod "7603959c-91b2-462c-81e7-98e0799baebf" (UID: "7603959c-91b2-462c-81e7-98e0799baebf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:43:25.962590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:25.962556    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:26.043572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.043538    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7603959c-91b2-462c-81e7-98e0799baebf-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:26.043572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.043564    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7603959c-91b2-462c-81e7-98e0799baebf-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:26.043572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.043572    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvpf6\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-kube-api-access-xvpf6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:26.043572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.043581    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7603959c-91b2-462c-81e7-98e0799baebf-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:26.139884 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.139835    4577 generic.go:356] "Generic (PLEG): container finished" podID="7603959c-91b2-462c-81e7-98e0799baebf" containerID="c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308" exitCode=0
Oct 30 01:43:26.140048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.139902    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" event={"ID":"7603959c-91b2-462c-81e7-98e0799baebf","Type":"ContainerDied","Data":"c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308"}
Oct 30 01:43:26.140048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.139924    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"
Oct 30 01:43:26.140048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.139962    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9" event={"ID":"7603959c-91b2-462c-81e7-98e0799baebf","Type":"ContainerDied","Data":"abe03530be797a6bc7ad394ae7a23f7d43db53176e3cbfa016da0d8c59aef54e"}
Oct 30 01:43:26.140048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.139986    4577 scope.go:122] "RemoveContainer" containerID="c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308"
Oct 30 01:43:26.151627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.151608    4577 scope.go:122] "RemoveContainer" containerID="2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"
Oct 30 01:43:26.161804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.161787    4577 scope.go:122] "RemoveContainer" containerID="c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308"
Oct 30 01:43:26.162082 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:26.162063    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308\": container with ID starting with c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308 not found: ID does not exist" containerID="c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308"
Oct 30 01:43:26.162151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.162093    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308"} err="failed to get container status \"c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308\": rpc error: code = NotFound desc = could not find container \"c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308\": container with ID starting with c7c0aca1e0922992179dff3e3f536c1be46acb5dbfff13ec4f39c3ad156ec308 not found: ID does not exist"
Oct 30 01:43:26.162151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.162112    4577 scope.go:122] "RemoveContainer" containerID="2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"
Oct 30 01:43:26.162356 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:26.162341    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1\": container with ID starting with 2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1 not found: ID does not exist" containerID="2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"
Oct 30 01:43:26.162414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.162360    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1"} err="failed to get container status \"2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1\": rpc error: code = NotFound desc = could not find container \"2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1\": container with ID starting with 2f39cf78430e9647867f4dde7296f86cdc29f391132cbd8ac77933431b4071c1 not found: ID does not exist"
Oct 30 01:43:26.200306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.200282    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"]
Oct 30 01:43:26.206934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:26.206912    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-190407-openshift-default-7b55ff7fcf-v4mz9"]
Oct 30 01:43:27.015409 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:27.015374    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:43:27.610341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:27.610306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:28.112958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:28.112930    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7603959c-91b2-462c-81e7-98e0799baebf" path="/var/lib/kubelet/pods/7603959c-91b2-462c-81e7-98e0799baebf/volumes"
Oct 30 01:43:29.297265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:29.297217    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:31.138093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:31.138054    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:33.129671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:33.129640    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:34.944034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:34.944002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:35.087186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.087161    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:43:35.202979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.202934    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume\") pod \"a276ce33-eadf-4310-b249-9b1d35c82eeb\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") "
Oct 30 01:43:35.203217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.203196    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp\" (UniqueName: \"kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp\") pod \"a276ce33-eadf-4310-b249-9b1d35c82eeb\" (UID: \"a276ce33-eadf-4310-b249-9b1d35c82eeb\") "
Oct 30 01:43:35.203427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.203398    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume" pod "a276ce33-eadf-4310-b249-9b1d35c82eeb" (UID: "a276ce33-eadf-4310-b249-9b1d35c82eeb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:43:35.204080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.204051    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a276ce33-eadf-4310-b249-9b1d35c82eeb-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:35.206180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.206143    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp" pod "a276ce33-eadf-4310-b249-9b1d35c82eeb" (UID: "a276ce33-eadf-4310-b249-9b1d35c82eeb"). InnerVolumeSpecName "kube-api-access-q7vqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:43:35.258712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.258678    4577 generic.go:356] "Generic (PLEG): container finished" podID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerID="12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1" exitCode=0
Oct 30 01:43:35.258921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.258726    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"
Oct 30 01:43:35.258921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.258760    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" event={"ID":"a276ce33-eadf-4310-b249-9b1d35c82eeb","Type":"ContainerDied","Data":"12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1"}
Oct 30 01:43:35.258921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.258817    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" event={"ID":"a276ce33-eadf-4310-b249-9b1d35c82eeb","Type":"ContainerDied","Data":"71e57518422a2b3b0949ab8e237ec7c445cd749aa66966bebe797c9eb9c7148a"}
Oct 30 01:43:35.258921 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.258858    4577 scope.go:122] "RemoveContainer" containerID="12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1"
Oct 30 01:43:35.271504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.271482    4577 scope.go:122] "RemoveContainer" containerID="12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1"
Oct 30 01:43:35.271775 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:35.271752    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1\": container with ID starting with 12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1 not found: ID does not exist" containerID="12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1"
Oct 30 01:43:35.271876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.271787    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1"} err="failed to get container status \"12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1\": rpc error: code = NotFound desc = could not find container \"12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1\": container with ID starting with 12825f0f97fee9f9d769db1a36c914bc3c8cbdb2fdd307dc3651bc876dced2b1 not found: ID does not exist"
Oct 30 01:43:35.305982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.305910    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7vqp\" (UniqueName: \"kubernetes.io/projected/a276ce33-eadf-4310-b249-9b1d35c82eeb-kube-api-access-q7vqp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:43:35.323181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.323143    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"]
Oct 30 01:43:35.332588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.332560    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm"]
Oct 30 01:43:35.977009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:35.976963    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-ab0a2c-7f79cf5664-qwfsm" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.252:50052\" within 1s: context deadline exceeded"
Oct 30 01:43:35.977439 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:43:35.977059    4577 logging.go:55] [core] [Channel #9280 SubChannel #9281]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.252:50052", ServerName: "10.134.0.252:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.252:50052: operation was canceled"
Oct 30 01:43:36.114798 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:36.114767    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" path="/var/lib/kubelet/pods/a276ce33-eadf-4310-b249-9b1d35c82eeb/volumes"
Oct 30 01:43:36.757645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:36.757598    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:38.618920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:38.618877    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:40.324101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:40.324031    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:40.456720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:40.456684    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:40.890059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:40.890021    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:42.327516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:42.327475    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:44.122318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:44.122284    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:45.960408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:45.960375    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:47.756393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:47.756359    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:49.563320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:49.563281    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:51.338297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:51.338258    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:53.337612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:53.337571    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:55.214932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:55.214897    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:55.476664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:55.476574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:57.021021 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:43:57.020975    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:43:57.538337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:57.538301    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:43:59.539765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:43:59.539730    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:01.210643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:01.210610    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:01.745791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:01.745752    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:01.915277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:01.915231    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:02.413707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:02.413678    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:03.424365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:03.424335    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:04.028305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:04.028270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:06.063391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:06.063352    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:06.381396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:06.381286    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:06.531534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:06.531495    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:07.459042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.459008    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"]
Oct 30 01:44:07.460092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460064    4577 cpu_manager.go:414] "containerMap: removing container" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy"
Oct 30 01:44:07.460092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460092    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460110    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460117    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460138    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460146    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460181    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460188    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9379b6ab-f35d-4d33-bca6-9c9073c290f9" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460200    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460207    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460223    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy"
Oct 30 01:44:07.460252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460231    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460277    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6fbc85a4-749c-4b50-be11-cf23de742321" containerName="revoke-keys"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460285    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fbc85a4-749c-4b50-be11-cf23de742321" containerName="revoke-keys"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460315    4577 cpu_manager.go:414] "containerMap: removing container" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460322    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460484    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="59836d05-c574-436c-9348-2d05f1faf029" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460499    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460508    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0856659-4315-48f2-be0f-def689ca8af3" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460519    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a276ce33-eadf-4310-b249-9b1d35c82eeb" containerName="payload-processing"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460535    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460545    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7603959c-91b2-462c-81e7-98e0799baebf" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460557    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy"
Oct 30 01:44:07.461174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.460572    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fbc85a4-749c-4b50-be11-cf23de742321" containerName="revoke-keys"
Oct 30 01:44:07.465142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.465118    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.511836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.511799    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"]
Oct 30 01:44:07.628566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628532    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.628566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628570    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.628826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628587    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.628826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628673    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.628826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628790    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.628826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628817    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svbhc\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.629088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.628982    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.629088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.629050    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.629187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.629174    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.730979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.730952    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.730988    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731005    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731031    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731100    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731118    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svbhc\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731176    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731224    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731338    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731447    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731505    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731725 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731567    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.731797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.731754    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.732054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.732030    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.733799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.733773    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.734013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.733993    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.756458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.756424    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.757038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.756999    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svbhc\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc\") pod \"maas-default-gateway-openshift-default-fcb75c45f-x8q6j\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:07.808417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:07.808386    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:08.009405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.009306    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:08.162644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.162507    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"]
Oct 30 01:44:08.165335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.165295    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:44:08.165427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.165379    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:44:08.165427 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.165411    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:44:08.634114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.634074    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" event={"ID":"8a4425a4-df44-4ba0-adce-5e733e2e6285","Type":"ContainerStarted","Data":"8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e"}
Oct 30 01:44:08.634613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.634129    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" event={"ID":"8a4425a4-df44-4ba0-adce-5e733e2e6285","Type":"ContainerStarted","Data":"7b87b0cdf04bd321064602ac8af17b3fba2a535428d1e6dc66a4e7a29772df92"}
Oct 30 01:44:08.694818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.694754    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podStartSLOduration=1.694734484 podStartE2EDuration="1.694734484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:44:08.692506698 +0000 UTC m=+3756.721816934" watchObservedRunningTime="2026-10-30 01:44:08.694734484 +0000 UTC m=+3756.724044716"
Oct 30 01:44:08.809006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:08.808976    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:09.754871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:09.753124    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:09.810922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:09.810887    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:09.811147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:09.810942    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:09.884096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:09.884057    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:10.096275 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:10.096158    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:10.245356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:10.236495    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:10.808949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:10.808918    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:10.809372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:10.808968    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:11.186628 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:11.186529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:11.312362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:11.312330    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:11.808924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:11.808889    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:11.809090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:11.808939    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:12.051375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:12.051339    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:12.179027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:12.178937    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:12.809714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:12.809680    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:12.810142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:12.809733    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:13.809225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:13.809177    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:13.809456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:13.809275    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:14.240897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:14.240838    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:14.433104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:14.433072    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:14.602424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:14.602336    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:14.809781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:14.809743    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:14.810017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:14.809812    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:15.809730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:15.809696    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused" start-of-body=
Oct 30 01:44:15.810229 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:15.809746    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": dial tcp 10.134.1.9:15021: connect: connection refused"
Oct 30 01:44:16.122132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:16.122055    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:17.411971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:17.411915    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:17.413175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:17.413153    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:17.413893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:17.413873    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:44:18.040964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:18.040788    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:19.773802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:19.773769    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:21.720416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:21.720383    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:22.480175 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.480139    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"]
Oct 30 01:44:22.480952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.480936    4577 cpu_manager.go:414] "containerMap: removing container" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy"
Oct 30 01:44:22.481033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.480952    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="434adec4-4a45-460c-a1c8-36cfbdd2092a" containerName="istio-proxy"
Oct 30 01:44:22.485093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.485073    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.508379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.508355    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"]
Oct 30 01:44:22.633378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633343    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633398    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633507    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633541    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633681    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633723    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633811    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg8qc\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633895    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.633966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.633939    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735095    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735146    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735216    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg8qc\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735299    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735355    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735432    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735481    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735542    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735557    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735573    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.735707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735641    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.736394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.735906    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.736394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.736088    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.736394 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.736235    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.737913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.737886    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.738244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.738224    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.761277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.761249    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg8qc\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.761405 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.761282    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token\") pod \"maas-default-gateway-openshift-default-fcb75c45f-69fm9\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:22.805276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:22.805246    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:23.151411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.151379    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"]
Oct 30 01:44:23.151656 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:44:23.151634    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0d7936_d648_4a3d_bf12_3ae6d46503e2.slice/crio-9bd0ca7e094245e49eeb124bc744dd7c3e37ac675fd3281d99a1512865fdce59 WatchSource:0}: Error finding container 9bd0ca7e094245e49eeb124bc744dd7c3e37ac675fd3281d99a1512865fdce59: Status 404 returned error can't find the container with id 9bd0ca7e094245e49eeb124bc744dd7c3e37ac675fd3281d99a1512865fdce59
Oct 30 01:44:23.155304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.155273    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:44:23.155386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.155339    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:44:23.155430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.155386    4577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 01:44:23.721543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.721506    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:23.872118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.870319    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" event={"ID":"6c0d7936-d648-4a3d-bf12-3ae6d46503e2","Type":"ContainerStarted","Data":"b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261"}
Oct 30 01:44:23.872118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.870389    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" event={"ID":"6c0d7936-d648-4a3d-bf12-3ae6d46503e2","Type":"ContainerStarted","Data":"9bd0ca7e094245e49eeb124bc744dd7c3e37ac675fd3281d99a1512865fdce59"}
Oct 30 01:44:23.972927 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:23.970548    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podStartSLOduration=1.9705270449999999 podStartE2EDuration="1.970527045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:44:23.936902542 +0000 UTC m=+3771.966212772" watchObservedRunningTime="2026-10-30 01:44:23.970527045 +0000 UTC m=+3771.999837277"
Oct 30 01:44:24.806188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:24.806153    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:24.807901 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:24.807875    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:24.808029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:24.807926    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:25.594364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:25.594333    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:25.806111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:25.806072    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:25.806307 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:25.806125    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:26.097638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:26.097601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:26.806095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:26.806048    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:26.806588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:26.806108    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:27.016868 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:44:27.016808    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:44:27.599153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:27.599122    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:27.806188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:27.806144    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:27.806602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:27.806207    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:28.805732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:28.805700    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:28.805975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:28.805772    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:29.461500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:29.461463    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:29.806535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:29.806493    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:29.806726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:29.806579    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:30.805717 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:30.805678    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused" start-of-body=
Oct 30 01:44:30.806279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:30.805755    4577 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": dial tcp 10.134.1.10:15021: connect: connection refused"
Oct 30 01:44:31.306488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:31.306453    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:31.992194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:31.992162    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:31.996177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:31.996154    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:32.000487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:32.000469    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:44:33.143096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:33.143064    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:34.967476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:34.967445    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:36.714947 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:36.714914    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:38.268173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:38.268132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:38.533106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:38.533021    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:40.463872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:40.461564    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:41.892110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:41.892065    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"]
Oct 30 01:44:41.892589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:41.892346    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q" podUID="8b305fa1-a45e-4810-810b-e3f661117406" containerName="maas-api" containerID="cri-o://a665b176589ed362951645562ae9f15182ec40e4f335dca53e100e698a7dad4c" gracePeriod=30
Oct 30 01:44:41.967025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:41.966986    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:42.050386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.050352    4577 generic.go:356] "Generic (PLEG): container finished" podID="8b305fa1-a45e-4810-810b-e3f661117406" containerID="a665b176589ed362951645562ae9f15182ec40e4f335dca53e100e698a7dad4c" exitCode=0
Oct 30 01:44:42.050591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.050431    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q" event={"ID":"8b305fa1-a45e-4810-810b-e3f661117406","Type":"ContainerDied","Data":"a665b176589ed362951645562ae9f15182ec40e4f335dca53e100e698a7dad4c"}
Oct 30 01:44:42.375090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.375058    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:44:42.442476 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.442425    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls\") pod \"8b305fa1-a45e-4810-810b-e3f661117406\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") "
Oct 30 01:44:42.442810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.442786    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca\") pod \"8b305fa1-a45e-4810-810b-e3f661117406\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") "
Oct 30 01:44:42.443147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.443129    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs\") pod \"8b305fa1-a45e-4810-810b-e3f661117406\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") "
Oct 30 01:44:42.443281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.443266    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7\" (UniqueName: \"kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7\") pod \"8b305fa1-a45e-4810-810b-e3f661117406\" (UID: \"8b305fa1-a45e-4810-810b-e3f661117406\") "
Oct 30 01:44:42.451874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.450816    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca" pod "8b305fa1-a45e-4810-810b-e3f661117406" (UID: "8b305fa1-a45e-4810-810b-e3f661117406"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:44:42.452367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.452344    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b305fa1-a45e-4810-810b-e3f661117406-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:42.485865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.484018    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs" pod "8b305fa1-a45e-4810-810b-e3f661117406" (UID: "8b305fa1-a45e-4810-810b-e3f661117406"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:44:42.485865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.485136    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls" pod "8b305fa1-a45e-4810-810b-e3f661117406" (UID: "8b305fa1-a45e-4810-810b-e3f661117406"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:44:42.515106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.514638    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7" pod "8b305fa1-a45e-4810-810b-e3f661117406" (UID: "8b305fa1-a45e-4810-810b-e3f661117406"). InnerVolumeSpecName "kube-api-access-7xmv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:44:42.515712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.515547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:42.643166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.642891    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:42.643166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.642940    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xmv7\" (UniqueName: \"kubernetes.io/projected/8b305fa1-a45e-4810-810b-e3f661117406-kube-api-access-7xmv7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:42.643166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.642954    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b305fa1-a45e-4810-810b-e3f661117406-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:42.673960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:42.673391    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:43.063761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:43.063730    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"
Oct 30 01:44:43.063761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:43.063749    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q" event={"ID":"8b305fa1-a45e-4810-810b-e3f661117406","Type":"ContainerDied","Data":"75494021dafdcc6e7471eaf508b2ee8957c588994c3efc6096fbf978c1c918b2"}
Oct 30 01:44:43.064366 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:43.064205    4577 scope.go:122] "RemoveContainer" containerID="a665b176589ed362951645562ae9f15182ec40e4f335dca53e100e698a7dad4c"
Oct 30 01:44:43.127182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:43.127146    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"]
Oct 30 01:44:43.137193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:43.137168    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-db4dd2-7f7c8ffdb8-tt99q"]
Oct 30 01:44:43.179424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:43.179397    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:44.201225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:44.201189    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b305fa1-a45e-4810-810b-e3f661117406" path="/var/lib/kubelet/pods/8b305fa1-a45e-4810-810b-e3f661117406/volumes"
Oct 30 01:44:44.449965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:44.449913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:45.013752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:45.013714    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:45.173688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:45.173608    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:45.301925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:45.300933    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:45.655801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:45.655709    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:46.917788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:46.917749    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:47.449124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:47.449093    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:49.241318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:49.241282    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:50.230579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:50.230540    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:50.357320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:50.357289    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:50.550747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:50.550651    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"]
Oct 30 01:44:50.551002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:50.550975    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" containerID="cri-o://6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb" gracePeriod=30
Oct 30 01:44:50.578782 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:50.578743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:50.990499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:50.990466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:51.492570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:51.492532    4577 patch_prober.go:29] interesting pod/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:44:51.493068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:51.492595    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:44:52.671440 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:52.671408    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:52.789705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:52.789671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:53.046347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:53.046311    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:53.178284 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:53.178246    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:54.983744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:54.983710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:55.867872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.867822    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:44:55.986340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986302    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.986825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986407    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.986825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986449    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.986825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986542    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.986825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986613    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.986825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986737    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.986825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986780    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.987194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986909    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.987194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.986963    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo\") pod \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\" (UID: \"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460\") "
Oct 30 01:44:55.987194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.987084    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:44:55.987194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.987151    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:44:55.987739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.987712    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:55.987880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.987743    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:55.989054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.988855    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:44:55.990248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.989956    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:44:55.990248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.990218    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:44:55.993273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.993245    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:44:55.995626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.995585    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:44:55.997643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.997619    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:44:55.997926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:55.997906    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75" pod "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" (UID: "192c1a31-d1a2-4fb4-8c78-f2ae9e97e460"). InnerVolumeSpecName "kube-api-access-5dh75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:44:56.089439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089316    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.089439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089360    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.089439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089401    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dh75\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-kube-api-access-5dh75\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.089439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089413    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.089439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089427    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.089439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089438    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.089758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.089449    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:44:56.329906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.329829    4577 generic.go:356] "Generic (PLEG): container finished" podID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerID="6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb" exitCode=0
Oct 30 01:44:56.329906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.329897    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"
Oct 30 01:44:56.330146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.329879    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" event={"ID":"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460","Type":"ContainerDied","Data":"6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb"}
Oct 30 01:44:56.330146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.330017    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6" event={"ID":"192c1a31-d1a2-4fb4-8c78-f2ae9e97e460","Type":"ContainerDied","Data":"b7f937b1ef3efb7601dd7edb70ba4e5fd15201c9362c269d7916f712ce406dd4"}
Oct 30 01:44:56.330146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.330044    4577 scope.go:122] "RemoveContainer" containerID="6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb"
Oct 30 01:44:56.343803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.343785    4577 scope.go:122] "RemoveContainer" containerID="a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"
Oct 30 01:44:56.355748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.355673    4577 scope.go:122] "RemoveContainer" containerID="6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb"
Oct 30 01:44:56.356037 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:44:56.356017    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb\": container with ID starting with 6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb not found: ID does not exist" containerID="6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb"
Oct 30 01:44:56.356159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.356049    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb"} err="failed to get container status \"6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb\": rpc error: code = NotFound desc = could not find container \"6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb\": container with ID starting with 6ac0072cb5ee2f87ff7a09c6136bde00a650eccf8dd6b6aabe4852f291c520cb not found: ID does not exist"
Oct 30 01:44:56.356159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.356075    4577 scope.go:122] "RemoveContainer" containerID="a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"
Oct 30 01:44:56.356410 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:44:56.356388    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d\": container with ID starting with a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d not found: ID does not exist" containerID="a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"
Oct 30 01:44:56.356534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.356415    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d"} err="failed to get container status \"a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d\": rpc error: code = NotFound desc = could not find container \"a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d\": container with ID starting with a30716220e0b710625603366fbb52427993cb7cf3d3f88017e64c81beab8bc0d not found: ID does not exist"
Oct 30 01:44:56.391577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.391543    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"]
Oct 30 01:44:56.404988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:56.404964    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-db4dd2-openshift-default-7d4694c95-v4jq6"]
Oct 30 01:44:57.016382 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:44:57.016343    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:44:57.039533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:57.039503    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:44:58.115403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:58.115373    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" path="/var/lib/kubelet/pods/192c1a31-d1a2-4fb4-8c78-f2ae9e97e460/volumes"
Oct 30 01:44:58.812131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:44:58.812049    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:00.221925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.221868    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"]
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.222954    4577 cpu_manager.go:414] "containerMap: removing container" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy"
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.222978    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy"
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.223042    4577 cpu_manager.go:414] "containerMap: removing container" podUID="8b305fa1-a45e-4810-810b-e3f661117406" containerName="maas-api"
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.223050    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b305fa1-a45e-4810-810b-e3f661117406" containerName="maas-api"
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.223238    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b305fa1-a45e-4810-810b-e3f661117406" containerName="maas-api"
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.223258    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy"
Oct 30 01:45:00.229416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.223276    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy"
Oct 30 01:45:00.229923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.229632    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.233470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.233444    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"]
Oct 30 01:45:00.234664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.234638    4577 cpu_manager.go:414] "containerMap: removing container" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy"
Oct 30 01:45:00.234824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.234808    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="192c1a31-d1a2-4fb4-8c78-f2ae9e97e460" containerName="istio-proxy"
Oct 30 01:45:00.240664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.240632    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.258295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.258265    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"]
Oct 30 01:45:00.265041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.264997    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"]
Oct 30 01:45:00.270775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.270747    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"]
Oct 30 01:45:00.276899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.276875    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.293544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.293402    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:45:00.300320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.300296    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"]
Oct 30 01:45:00.311680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.311654    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"]
Oct 30 01:45:00.318135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.318102    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"]
Oct 30 01:45:00.325505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.325472    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.327146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.327034    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"]
Oct 30 01:45:00.327323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.327114    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.352564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.352540    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"]
Oct 30 01:45:00.363572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.363548    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"]
Oct 30 01:45:00.370414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.370393    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.378640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.378600    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.378940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.378886    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct8bd\" (UniqueName: \"kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.379151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.379133    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.379273 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.379258    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrl4w\" (UniqueName: \"kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.379433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.379419    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-579qc\" (UniqueName: \"kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc\") pod \"maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.380606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.380582    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.381532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.381506    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"]
Oct 30 01:45:00.385080 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.384960    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"]
Oct 30 01:45:00.392634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.392409    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.393808 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.393788    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"]
Oct 30 01:45:00.406170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.406149    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.421915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.420380    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"]
Oct 30 01:45:00.439436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.428920    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"]
Oct 30 01:45:00.448865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.448003    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"]
Oct 30 01:45:00.448865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.448046    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"]
Oct 30 01:45:00.448865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.448131    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.482915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.482888    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.483072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.482934    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.483072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.482981    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct8bd\" (UniqueName: \"kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.483072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483015    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsk9p\" (UniqueName: \"kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.483072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483043    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483077    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483093    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrl4w\" (UniqueName: \"kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483109    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483154    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-579qc\" (UniqueName: \"kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc\") pod \"maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483171    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q794\" (UniqueName: \"kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794\") pod \"maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483194    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk4td\" (UniqueName: \"kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483223    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483264    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.483310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483294    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp27b\" (UniqueName: \"kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.483910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.483888    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.485038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.484943    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.485160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.485136    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.521079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.521051    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrl4w\" (UniqueName: \"kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.521288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.521270    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct8bd\" (UniqueName: \"kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.521431 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.521412    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-579qc\" (UniqueName: \"kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc\") pod \"maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.585019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.584980    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.585362 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585337    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp27b\" (UniqueName: \"kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.585508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585488    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5dzp\" (UniqueName: \"kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.585631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585614    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888745-sg6fp\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.585749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585724    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wlqf\" (UniqueName: \"kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf\") pod \"maas-api-key-cleanup-29888745-sg6fp\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.585816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585771    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.585935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585919    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsk9p\" (UniqueName: \"kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.585977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585960    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.586016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.585969    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.586790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.586258    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.586790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.586344    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.586790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.586372    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q794\" (UniqueName: \"kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794\") pod \"maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.586790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.586415    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk4td\" (UniqueName: \"kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.586790 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.586561    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.587185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.587164    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.587403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.587383    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.596205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.596181    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:00.606381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.606356    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:00.612329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.612300    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp27b\" (UniqueName: \"kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.612544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.612520    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk4td\" (UniqueName: \"kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.612875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.612825    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q794\" (UniqueName: \"kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794\") pod \"maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.613619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.613601    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsk9p\" (UniqueName: \"kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.613775 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.613762    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:00.658114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.657882    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:00.677894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.675421    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:00.691990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.689558    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.691990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.689690    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5dzp\" (UniqueName: \"kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.691990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.689790    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888745-sg6fp\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.691990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.689866    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wlqf\" (UniqueName: \"kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf\") pod \"maas-api-key-cleanup-29888745-sg6fp\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.691990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.691749    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888745-sg6fp\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.692396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.692074    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.700153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.700130    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:00.724445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.724395    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wlqf\" (UniqueName: \"kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf\") pod \"maas-api-key-cleanup-29888745-sg6fp\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.744397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.744335    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5dzp\" (UniqueName: \"kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.760079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.759832    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:00.788382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.788296    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:00.795409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.793566    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:00.838869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:00.832329    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:01.123173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.123064    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"]
Oct 30 01:45:01.123680 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:45:01.123659    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88de8ed2_d09a_4715_85f4_35d8a9a51755.slice/crio-d25e9bf8f384339bdd13a3d7be65a40590bb7fc541cb7c43bb4d1f57f1e7726a WatchSource:0}: Error finding container d25e9bf8f384339bdd13a3d7be65a40590bb7fc541cb7c43bb4d1f57f1e7726a: Status 404 returned error can't find the container with id d25e9bf8f384339bdd13a3d7be65a40590bb7fc541cb7c43bb4d1f57f1e7726a
Oct 30 01:45:01.162118 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:45:01.161776    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48323c68_17e2_4c68_9169_f7a61b27411f.slice/crio-5d876f957c5a329e2089f0ec907fd98cc30633a65e61616fa2b68fba6f9768ab WatchSource:0}: Error finding container 5d876f957c5a329e2089f0ec907fd98cc30633a65e61616fa2b68fba6f9768ab: Status 404 returned error can't find the container with id 5d876f957c5a329e2089f0ec907fd98cc30633a65e61616fa2b68fba6f9768ab
Oct 30 01:45:01.169593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.169476    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"]
Oct 30 01:45:01.216417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.216387    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"]
Oct 30 01:45:01.285696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.285664    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"]
Oct 30 01:45:01.311989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.311963    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"]
Oct 30 01:45:01.350797 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:45:01.350745    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6050dd5_3567_411b_93cb_fe49d4fd7177.slice/crio-e3b3b9bd3646fe321a0abec63abb5d3376489ae43f61a0068d91d8af1e5dd949 WatchSource:0}: Error finding container e3b3b9bd3646fe321a0abec63abb5d3376489ae43f61a0068d91d8af1e5dd949: Status 404 returned error can't find the container with id e3b3b9bd3646fe321a0abec63abb5d3376489ae43f61a0068d91d8af1e5dd949
Oct 30 01:45:01.351251 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:45:01.351227    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf87902_7cb2_4647_aca3_33b9b7e65b6a.slice/crio-c2ab095a90f02dff3e0c99412a2b65fc5fe84f03ea86649d6b03004aad08ad1d WatchSource:0}: Error finding container c2ab095a90f02dff3e0c99412a2b65fc5fe84f03ea86649d6b03004aad08ad1d: Status 404 returned error can't find the container with id c2ab095a90f02dff3e0c99412a2b65fc5fe84f03ea86649d6b03004aad08ad1d
Oct 30 01:45:01.392357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.392317    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerStarted","Data":"d25e9bf8f384339bdd13a3d7be65a40590bb7fc541cb7c43bb4d1f57f1e7726a"}
Oct 30 01:45:01.393929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.393901    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerStarted","Data":"e8bef6cc976c1b29adcdb2d5bcb5a040a6d562ddff80d339c8586b6eae7e9bbb"}
Oct 30 01:45:01.395402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.395368    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerStarted","Data":"e3b3b9bd3646fe321a0abec63abb5d3376489ae43f61a0068d91d8af1e5dd949"}
Oct 30 01:45:01.396724 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.396702    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerStarted","Data":"c2ab095a90f02dff3e0c99412a2b65fc5fe84f03ea86649d6b03004aad08ad1d"}
Oct 30 01:45:01.398169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.398144    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerStarted","Data":"5d876f957c5a329e2089f0ec907fd98cc30633a65e61616fa2b68fba6f9768ab"}
Oct 30 01:45:01.418538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.418507    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"]
Oct 30 01:45:01.424477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.424453    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"]
Oct 30 01:45:01.460086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.460062    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"]
Oct 30 01:45:01.517516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.517488    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"]
Oct 30 01:45:01.767983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.764803    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:01.914781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:01.914623    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:02.490608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.490573    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerStarted","Data":"e831814d97c2911c68dfb71faadd8f348b94a7368cef339d891499d489f1cc6a"}
Oct 30 01:45:02.491083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.490639    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerStarted","Data":"de80a049dcf27d0d8c4cd357c25cc78e9fd0a1381ad5a1d8c3aa7102d081853a"}
Oct 30 01:45:02.494428 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.494395    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerStarted","Data":"ab37363d561b92d2d0973c6d34f1a6c6338f3b2149e5e8140d90f8fd26a86bed"}
Oct 30 01:45:02.497694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.497664    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerStarted","Data":"2a023da64a36f5b606a130a3d52514f352f6cf077cd6b9b60792ee634ec7781c"}
Oct 30 01:45:02.503191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.502277    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerStarted","Data":"a938091c47e76b984d3f45fe6121f9b5812465d1664c2f9c1069a696f0fb370c"}
Oct 30 01:45:02.503191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.502330    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerStarted","Data":"bc8277980d30cbffa27d6e7685be05073d95ba8d7509f64d600b12517befd911"}
Oct 30 01:45:02.515642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.514023    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerStarted","Data":"d385682c72eea9bdd002b7dcd152ea84a3763ee0c50b9d0507f457b35f92b785"}
Oct 30 01:45:02.518408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.518327    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerStarted","Data":"af1918515b7b881d94a4f3cdaec4b5c44dcd0f1a21d00b02b370b355a04a26ae"}
Oct 30 01:45:02.520929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.520829    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerStarted","Data":"a4776ee5bd51cdde29245b8a8010f1ceab3e3735a51855b4b5887784775a49c1"}
Oct 30 01:45:02.543866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.543805    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" podStartSLOduration=2.543791027 podStartE2EDuration="2.543791027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:02.536885731 +0000 UTC m=+3810.566195963" watchObservedRunningTime="2026-10-30 01:45:02.543791027 +0000 UTC m=+3810.573101287"
Oct 30 01:45:02.583089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.583038    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" podStartSLOduration=2.583023622 podStartE2EDuration="2.583023622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:02.581219315 +0000 UTC m=+3810.610529547" watchObservedRunningTime="2026-10-30 01:45:02.583023622 +0000 UTC m=+3810.612333854"
Oct 30 01:45:02.641206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.641106    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" podStartSLOduration=2.641093952 podStartE2EDuration="2.641093952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:02.635748708 +0000 UTC m=+3810.665058940" watchObservedRunningTime="2026-10-30 01:45:02.641093952 +0000 UTC m=+3810.670404183"
Oct 30 01:45:02.687132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.687085    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" podStartSLOduration=2.687070718 podStartE2EDuration="2.687070718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:02.685289548 +0000 UTC m=+3810.714599777" watchObservedRunningTime="2026-10-30 01:45:02.687070718 +0000 UTC m=+3810.716380946"
Oct 30 01:45:02.714710 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:02.714679    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddabe84af_0e14_4545_a2a7_1c13a0814000.slice/crio-conmon-2a023da64a36f5b606a130a3d52514f352f6cf077cd6b9b60792ee634ec7781c.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 01:45:02.736607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.736557    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" podStartSLOduration=2.736542557 podStartE2EDuration="2.736542557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:02.735287392 +0000 UTC m=+3810.764597624" watchObservedRunningTime="2026-10-30 01:45:02.736542557 +0000 UTC m=+3810.765852788"
Oct 30 01:45:02.837537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:02.837506    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:03.531533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.531498    4577 generic.go:356] "Generic (PLEG): container finished" podID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerID="e831814d97c2911c68dfb71faadd8f348b94a7368cef339d891499d489f1cc6a" exitCode=60
Oct 30 01:45:03.532046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.531552    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerDied","Data":"e831814d97c2911c68dfb71faadd8f348b94a7368cef339d891499d489f1cc6a"}
Oct 30 01:45:03.532046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.531955    4577 scope.go:122] "RemoveContainer" containerID="e831814d97c2911c68dfb71faadd8f348b94a7368cef339d891499d489f1cc6a"
Oct 30 01:45:03.533596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.533570    4577 generic.go:356] "Generic (PLEG): container finished" podID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerID="ab37363d561b92d2d0973c6d34f1a6c6338f3b2149e5e8140d90f8fd26a86bed" exitCode=60
Oct 30 01:45:03.533714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.533651    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerDied","Data":"ab37363d561b92d2d0973c6d34f1a6c6338f3b2149e5e8140d90f8fd26a86bed"}
Oct 30 01:45:03.533945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.533930    4577 scope.go:122] "RemoveContainer" containerID="ab37363d561b92d2d0973c6d34f1a6c6338f3b2149e5e8140d90f8fd26a86bed"
Oct 30 01:45:03.551123 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.551088    4577 generic.go:356] "Generic (PLEG): container finished" podID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerID="2a023da64a36f5b606a130a3d52514f352f6cf077cd6b9b60792ee634ec7781c" exitCode=60
Oct 30 01:45:03.551367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.551345    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerDied","Data":"2a023da64a36f5b606a130a3d52514f352f6cf077cd6b9b60792ee634ec7781c"}
Oct 30 01:45:03.552000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.551926    4577 scope.go:122] "RemoveContainer" containerID="2a023da64a36f5b606a130a3d52514f352f6cf077cd6b9b60792ee634ec7781c"
Oct 30 01:45:03.554187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.554157    4577 generic.go:356] "Generic (PLEG): container finished" podID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerID="99b5ca9bb5f4ce4509b7beec88b89ae889fa7e854cbf2396ed4314e590e44151" exitCode=60
Oct 30 01:45:03.554283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.554272    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerDied","Data":"99b5ca9bb5f4ce4509b7beec88b89ae889fa7e854cbf2396ed4314e590e44151"}
Oct 30 01:45:03.555531 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.555500    4577 scope.go:122] "RemoveContainer" containerID="99b5ca9bb5f4ce4509b7beec88b89ae889fa7e854cbf2396ed4314e590e44151"
Oct 30 01:45:03.557279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.557256    4577 generic.go:356] "Generic (PLEG): container finished" podID="e45dc766-3e15-4a91-a753-de02ff246993" containerID="a938091c47e76b984d3f45fe6121f9b5812465d1664c2f9c1069a696f0fb370c" exitCode=60
Oct 30 01:45:03.557425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.557407    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerDied","Data":"a938091c47e76b984d3f45fe6121f9b5812465d1664c2f9c1069a696f0fb370c"}
Oct 30 01:45:03.559693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.557932    4577 scope.go:122] "RemoveContainer" containerID="a938091c47e76b984d3f45fe6121f9b5812465d1664c2f9c1069a696f0fb370c"
Oct 30 01:45:03.561956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.561927    4577 generic.go:356] "Generic (PLEG): container finished" podID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerID="69a6ea322e6f747c535b7c09d9db79afd27ff927feaf494e11bbbf67dbf10b86" exitCode=60
Oct 30 01:45:03.562061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.562008    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerDied","Data":"69a6ea322e6f747c535b7c09d9db79afd27ff927feaf494e11bbbf67dbf10b86"}
Oct 30 01:45:03.562324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.562244    4577 scope.go:122] "RemoveContainer" containerID="69a6ea322e6f747c535b7c09d9db79afd27ff927feaf494e11bbbf67dbf10b86"
Oct 30 01:45:03.572613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.569777    4577 generic.go:356] "Generic (PLEG): container finished" podID="48323c68-17e2-4c68-9169-f7a61b27411f" containerID="d385682c72eea9bdd002b7dcd152ea84a3763ee0c50b9d0507f457b35f92b785" exitCode=60
Oct 30 01:45:03.572613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.569888    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerDied","Data":"d385682c72eea9bdd002b7dcd152ea84a3763ee0c50b9d0507f457b35f92b785"}
Oct 30 01:45:03.572613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.570127    4577 scope.go:122] "RemoveContainer" containerID="d385682c72eea9bdd002b7dcd152ea84a3763ee0c50b9d0507f457b35f92b785"
Oct 30 01:45:03.580133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.580106    4577 generic.go:356] "Generic (PLEG): container finished" podID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerID="28a25cdd0f1de1ffdc6ac08bc2f1ff4bb2f3c8db6044a83371eb2416668cfae6" exitCode=60
Oct 30 01:45:03.580550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.580529    4577 scope.go:122] "RemoveContainer" containerID="28a25cdd0f1de1ffdc6ac08bc2f1ff4bb2f3c8db6044a83371eb2416668cfae6"
Oct 30 01:45:03.580648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.580566    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerDied","Data":"28a25cdd0f1de1ffdc6ac08bc2f1ff4bb2f3c8db6044a83371eb2416668cfae6"}
Oct 30 01:45:03.598299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.598268    4577 generic.go:356] "Generic (PLEG): container finished" podID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerID="2c34983c38e94ff96168f51001193c467004929cc66226ea97524caece545c88" exitCode=60
Oct 30 01:45:03.598409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.598330    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerDied","Data":"2c34983c38e94ff96168f51001193c467004929cc66226ea97524caece545c88"}
Oct 30 01:45:03.598772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:03.598739    4577 scope.go:122] "RemoveContainer" containerID="2c34983c38e94ff96168f51001193c467004929cc66226ea97524caece545c88"
Oct 30 01:45:04.126614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.126517    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"]
Oct 30 01:45:04.269980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.269928    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:04.591046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.591010    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:04.609570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.609533    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerStarted","Data":"7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090"}
Oct 30 01:45:04.611890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.611856    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerStarted","Data":"ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c"}
Oct 30 01:45:04.614363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.614339    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerStarted","Data":"1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8"}
Oct 30 01:45:04.616484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.616458    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerStarted","Data":"8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed"}
Oct 30 01:45:04.618505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.618480    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerStarted","Data":"2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f"}
Oct 30 01:45:04.620684 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.620646    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerStarted","Data":"91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153"}
Oct 30 01:45:04.622826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.622800    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerStarted","Data":"68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1"}
Oct 30 01:45:04.624964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.624938    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerStarted","Data":"387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c"}
Oct 30 01:45:04.626885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.626823    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerStarted","Data":"41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc"}
Oct 30 01:45:04.764257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.764192    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" podStartSLOduration=4.764171883 podStartE2EDuration="4.764171883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:04.758222269 +0000 UTC m=+3812.787532501" watchObservedRunningTime="2026-10-30 01:45:04.764171883 +0000 UTC m=+3812.793482116"
Oct 30 01:45:04.804827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.804761    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" podStartSLOduration=4.804746088 podStartE2EDuration="4.804746088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:04.801101374 +0000 UTC m=+3812.830411605" watchObservedRunningTime="2026-10-30 01:45:04.804746088 +0000 UTC m=+3812.834056318"
Oct 30 01:45:04.879872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.872274    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" podStartSLOduration=4.872254412 podStartE2EDuration="4.872254412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:04.856172617 +0000 UTC m=+3812.885482849" watchObservedRunningTime="2026-10-30 01:45:04.872254412 +0000 UTC m=+3812.901564644"
Oct 30 01:45:05.082356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:04.990481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:05.195591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.195512    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" podStartSLOduration=5.195488346 podStartE2EDuration="5.195488346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:45:05.142291184 +0000 UTC m=+3813.171601417" watchObservedRunningTime="2026-10-30 01:45:05.195488346 +0000 UTC m=+3813.224798579"
Oct 30 01:45:05.637967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.637925    4577 generic.go:356] "Generic (PLEG): container finished" podID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerID="8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed" exitCode=60
Oct 30 01:45:05.638456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.637968    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerDied","Data":"8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed"}
Oct 30 01:45:05.638456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.638036    4577 scope.go:122] "RemoveContainer" containerID="99b5ca9bb5f4ce4509b7beec88b89ae889fa7e854cbf2396ed4314e590e44151"
Oct 30 01:45:05.638456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.638400    4577 scope.go:122] "RemoveContainer" containerID="8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed"
Oct 30 01:45:05.638838 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.638817    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn_odh-ai-gateway-infra(f6050dd5-3567-411b-93cb-fe49d4fd7177)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177"
Oct 30 01:45:05.640276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.640255    4577 generic.go:356] "Generic (PLEG): container finished" podID="e45dc766-3e15-4a91-a753-de02ff246993" containerID="2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f" exitCode=60
Oct 30 01:45:05.640276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.640287    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerDied","Data":"2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f"}
Oct 30 01:45:05.640644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.640630    4577 scope.go:122] "RemoveContainer" containerID="2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f"
Oct 30 01:45:05.641032 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.641011    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g_odh-ai-gateway-infra(e45dc766-3e15-4a91-a753-de02ff246993)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" podUID="e45dc766-3e15-4a91-a753-de02ff246993"
Oct 30 01:45:05.642299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.642279    4577 generic.go:356] "Generic (PLEG): container finished" podID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerID="91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153" exitCode=60
Oct 30 01:45:05.642388 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.642365    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerDied","Data":"91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153"}
Oct 30 01:45:05.642629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.642612    4577 scope.go:122] "RemoveContainer" containerID="91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153"
Oct 30 01:45:05.643012 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.642986    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r_odh-ai-gateway-infra(0cf87902-7cb2-4647-aca3-33b9b7e65b6a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a"
Oct 30 01:45:05.644355 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.644336    4577 generic.go:356] "Generic (PLEG): container finished" podID="48323c68-17e2-4c68-9169-f7a61b27411f" containerID="68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1" exitCode=60
Oct 30 01:45:05.644454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.644416    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerDied","Data":"68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1"}
Oct 30 01:45:05.644674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.644658    4577 scope.go:122] "RemoveContainer" containerID="68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1"
Oct 30 01:45:05.645003 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.644984    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb_odh-ai-gateway-infra(48323c68-17e2-4c68-9169-f7a61b27411f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" podUID="48323c68-17e2-4c68-9169-f7a61b27411f"
Oct 30 01:45:05.646621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.646555    4577 generic.go:356] "Generic (PLEG): container finished" podID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerID="387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c" exitCode=60
Oct 30 01:45:05.646621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.646578    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerDied","Data":"387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c"}
Oct 30 01:45:05.646912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.646895    4577 scope.go:122] "RemoveContainer" containerID="387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c"
Oct 30 01:45:05.647205 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.647185    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn_odh-ai-gateway-infra(1baf66d7-a06c-41b0-af3a-47d379b7ae30)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30"
Oct 30 01:45:05.649149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.649130    4577 generic.go:356] "Generic (PLEG): container finished" podID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerID="41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc" exitCode=60
Oct 30 01:45:05.649259 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.649216    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerDied","Data":"41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc"}
Oct 30 01:45:05.649773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.649754    4577 scope.go:122] "RemoveContainer" containerID="41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc"
Oct 30 01:45:05.650109 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.650083    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888745-sg6fp_odh-ai-gateway-infra(a251e7e1-745e-4870-9e8c-0e1c3513d789)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789"
Oct 30 01:45:05.651528 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.651507    4577 generic.go:356] "Generic (PLEG): container finished" podID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerID="7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090" exitCode=60
Oct 30 01:45:05.651612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.651563    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerDied","Data":"7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090"}
Oct 30 01:45:05.651868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.651824    4577 scope.go:122] "RemoveContainer" containerID="7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090"
Oct 30 01:45:05.652175 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.652152    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb_odh-ai-gateway-infra(2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c"
Oct 30 01:45:05.653754 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.653736    4577 generic.go:356] "Generic (PLEG): container finished" podID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerID="ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c" exitCode=60
Oct 30 01:45:05.653867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.653818    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerDied","Data":"ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c"}
Oct 30 01:45:05.654068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.654050    4577 scope.go:122] "RemoveContainer" containerID="ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c"
Oct 30 01:45:05.654439 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.654402    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm_odh-ai-gateway-infra(88de8ed2-d09a-4715-85f4-35d8a9a51755)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755"
Oct 30 01:45:05.655533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.655511    4577 scope.go:122] "RemoveContainer" containerID="a938091c47e76b984d3f45fe6121f9b5812465d1664c2f9c1069a696f0fb370c"
Oct 30 01:45:05.656287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.656267    4577 generic.go:356] "Generic (PLEG): container finished" podID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerID="1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8" exitCode=60
Oct 30 01:45:05.656415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.656308    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerDied","Data":"1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8"}
Oct 30 01:45:05.656562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.656547    4577 scope.go:122] "RemoveContainer" containerID="1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8"
Oct 30 01:45:05.656827 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:05.656809    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k_odh-ai-gateway-infra(dabe84af-0e14-4545-a2a7-1c13a0814000)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000"
Oct 30 01:45:05.670855 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.670826    4577 scope.go:122] "RemoveContainer" containerID="69a6ea322e6f747c535b7c09d9db79afd27ff927feaf494e11bbbf67dbf10b86"
Oct 30 01:45:05.686095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.686078    4577 scope.go:122] "RemoveContainer" containerID="d385682c72eea9bdd002b7dcd152ea84a3763ee0c50b9d0507f457b35f92b785"
Oct 30 01:45:05.701449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.701425    4577 scope.go:122] "RemoveContainer" containerID="28a25cdd0f1de1ffdc6ac08bc2f1ff4bb2f3c8db6044a83371eb2416668cfae6"
Oct 30 01:45:05.716679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.716659    4577 scope.go:122] "RemoveContainer" containerID="2c34983c38e94ff96168f51001193c467004929cc66226ea97524caece545c88"
Oct 30 01:45:05.747126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.747090    4577 scope.go:122] "RemoveContainer" containerID="e831814d97c2911c68dfb71faadd8f348b94a7368cef339d891499d489f1cc6a"
Oct 30 01:45:05.776178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.776143    4577 scope.go:122] "RemoveContainer" containerID="ab37363d561b92d2d0973c6d34f1a6c6338f3b2149e5e8140d90f8fd26a86bed"
Oct 30 01:45:05.798290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:05.798268    4577 scope.go:122] "RemoveContainer" containerID="2a023da64a36f5b606a130a3d52514f352f6cf077cd6b9b60792ee634ec7781c"
Oct 30 01:45:06.670565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.670533    4577 scope.go:122] "RemoveContainer" containerID="91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153"
Oct 30 01:45:06.671116 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.670989    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r_odh-ai-gateway-infra(0cf87902-7cb2-4647-aca3-33b9b7e65b6a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a"
Oct 30 01:45:06.672563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.672540    4577 scope.go:122] "RemoveContainer" containerID="68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1"
Oct 30 01:45:06.672980 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.672953    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb_odh-ai-gateway-infra(48323c68-17e2-4c68-9169-f7a61b27411f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" podUID="48323c68-17e2-4c68-9169-f7a61b27411f"
Oct 30 01:45:06.674483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.674453    4577 scope.go:122] "RemoveContainer" containerID="387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c"
Oct 30 01:45:06.674829 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.674808    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn_odh-ai-gateway-infra(1baf66d7-a06c-41b0-af3a-47d379b7ae30)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30"
Oct 30 01:45:06.676501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.676481    4577 scope.go:122] "RemoveContainer" containerID="41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc"
Oct 30 01:45:06.676792 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.676772    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888745-sg6fp_odh-ai-gateway-infra(a251e7e1-745e-4870-9e8c-0e1c3513d789)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789"
Oct 30 01:45:06.678473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.678453    4577 scope.go:122] "RemoveContainer" containerID="7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090"
Oct 30 01:45:06.678732 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.678701    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb_odh-ai-gateway-infra(2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c"
Oct 30 01:45:06.680420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.680403    4577 scope.go:122] "RemoveContainer" containerID="ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c"
Oct 30 01:45:06.680704 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.680661    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm_odh-ai-gateway-infra(88de8ed2-d09a-4715-85f4-35d8a9a51755)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755"
Oct 30 01:45:06.682567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.682548    4577 scope.go:122] "RemoveContainer" containerID="1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8"
Oct 30 01:45:06.682797 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.682773    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k_odh-ai-gateway-infra(dabe84af-0e14-4545-a2a7-1c13a0814000)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000"
Oct 30 01:45:06.684552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.684535    4577 scope.go:122] "RemoveContainer" containerID="8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed"
Oct 30 01:45:06.684801 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.684773    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn_odh-ai-gateway-infra(f6050dd5-3567-411b-93cb-fe49d4fd7177)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177"
Oct 30 01:45:06.686308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.686291    4577 scope.go:122] "RemoveContainer" containerID="2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f"
Oct 30 01:45:06.686558 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:06.686536    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g_odh-ai-gateway-infra(e45dc766-3e15-4a91-a753-de02ff246993)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" podUID="e45dc766-3e15-4a91-a753-de02ff246993"
Oct 30 01:45:06.751646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.751615    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:06.871045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:06.871008    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:08.648003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:08.647969    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:09.126890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:09.126798    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" podUID="c340ad11-f797-4128-abea-4ee3c7181708" containerName="payload-pre-processing" containerID="cri-o://d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f" gracePeriod=25
Oct 30 01:45:10.440396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:10.440363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:12.240075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:12.240034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:13.463896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:13.462522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:14.140375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:14.140332    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:15.904214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:15.904172    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:16.104334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:16.104300    4577 scope.go:122] "RemoveContainer" containerID="1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8"
Oct 30 01:45:16.104566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:16.104534    4577 scope.go:122] "RemoveContainer" containerID="8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed"
Oct 30 01:45:16.753708 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:16.753671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:16.806332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:16.806284    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerStarted","Data":"75cf035b92189c13568127093a23dd3647fcf69f24a1327c443274da5f4fd47d"}
Oct 30 01:45:16.808990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:16.808935    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerStarted","Data":"5cdf993420fdd4f18c26175653bec069a92e91cbe76113b7afd156643b6fcb18"}
Oct 30 01:45:17.104946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.104829    4577 scope.go:122] "RemoveContainer" containerID="91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153"
Oct 30 01:45:17.105343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.105124    4577 scope.go:122] "RemoveContainer" containerID="ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c"
Oct 30 01:45:17.802079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.802034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:17.819231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.819199    4577 generic.go:356] "Generic (PLEG): container finished" podID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerID="5cdf993420fdd4f18c26175653bec069a92e91cbe76113b7afd156643b6fcb18" exitCode=60
Oct 30 01:45:17.819425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.819261    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerDied","Data":"5cdf993420fdd4f18c26175653bec069a92e91cbe76113b7afd156643b6fcb18"}
Oct 30 01:45:17.819425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.819341    4577 scope.go:122] "RemoveContainer" containerID="1d5a0f43a0bf2c9d96f7192601caa38a5545c2fdfbd87d5eb696d7bf3d2270c8"
Oct 30 01:45:17.819805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.819789    4577 scope.go:122] "RemoveContainer" containerID="5cdf993420fdd4f18c26175653bec069a92e91cbe76113b7afd156643b6fcb18"
Oct 30 01:45:17.820577 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:17.820247    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k_odh-ai-gateway-infra(dabe84af-0e14-4545-a2a7-1c13a0814000)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000"
Oct 30 01:45:17.821586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.821558    4577 generic.go:356] "Generic (PLEG): container finished" podID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerID="75cf035b92189c13568127093a23dd3647fcf69f24a1327c443274da5f4fd47d" exitCode=60
Oct 30 01:45:17.821701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.821686    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerDied","Data":"75cf035b92189c13568127093a23dd3647fcf69f24a1327c443274da5f4fd47d"}
Oct 30 01:45:17.822136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.822116    4577 scope.go:122] "RemoveContainer" containerID="75cf035b92189c13568127093a23dd3647fcf69f24a1327c443274da5f4fd47d"
Oct 30 01:45:17.822515 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:17.822493    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn_odh-ai-gateway-infra(f6050dd5-3567-411b-93cb-fe49d4fd7177)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177"
Oct 30 01:45:17.823878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.823816    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerStarted","Data":"c10591251b26aa364710be5304425507949a5b439b564a841626a095a9447ecd"}
Oct 30 01:45:17.826118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.826095    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerStarted","Data":"e5f34ff68b58ac6942d00472fe92e2d1d8c2184598c5791465a9d5ddb2b9977f"}
Oct 30 01:45:17.841651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.840551    4577 scope.go:122] "RemoveContainer" containerID="8e4c8521f568f7d698574b7c58da93dc96c713121cfe228df21ffa534470c4ed"
Oct 30 01:45:17.921926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.913063    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"]
Oct 30 01:45:17.978715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:17.978681    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"]
Oct 30 01:45:18.105214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.105042    4577 scope.go:122] "RemoveContainer" containerID="2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f"
Oct 30 01:45:18.105214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.105212    4577 scope.go:122] "RemoveContainer" containerID="387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c"
Oct 30 01:45:18.105816 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.105365    4577 scope.go:122] "RemoveContainer" containerID="68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1"
Oct 30 01:45:18.730373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.730337    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" podUID="c340ad11-f797-4128-abea-4ee3c7181708" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:45:18.842624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.842591    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerStarted","Data":"3290c1af30c54a256e4c1bc57cbf59f19627adba9b5a1f310cf67bebda3b1274"}
Oct 30 01:45:18.853902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.846785    4577 generic.go:356] "Generic (PLEG): container finished" podID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerID="c10591251b26aa364710be5304425507949a5b439b564a841626a095a9447ecd" exitCode=60
Oct 30 01:45:18.853902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.846888    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerDied","Data":"c10591251b26aa364710be5304425507949a5b439b564a841626a095a9447ecd"}
Oct 30 01:45:18.853902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.846936    4577 scope.go:122] "RemoveContainer" containerID="91c3ef555a9122331b4d4a6a45e556f399ee769153eefef7a0eeac71e6ba8153"
Oct 30 01:45:18.853902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.847415    4577 scope.go:122] "RemoveContainer" containerID="c10591251b26aa364710be5304425507949a5b439b564a841626a095a9447ecd"
Oct 30 01:45:18.853902 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:18.847808    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r_odh-ai-gateway-infra(0cf87902-7cb2-4647-aca3-33b9b7e65b6a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a"
Oct 30 01:45:18.853902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.853041    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerStarted","Data":"3a6c66cfadf7fad33456640a92d6159b498b961e4de3b39983e037029a6b39e9"}
Oct 30 01:45:18.858978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.858947    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerStarted","Data":"045693c670381beaab86d518f13259a5db27bdac3360b1272bdee1605df0e64d"}
Oct 30 01:45:18.862326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.861491    4577 generic.go:356] "Generic (PLEG): container finished" podID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerID="e5f34ff68b58ac6942d00472fe92e2d1d8c2184598c5791465a9d5ddb2b9977f" exitCode=60
Oct 30 01:45:18.862326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.861535    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerDied","Data":"e5f34ff68b58ac6942d00472fe92e2d1d8c2184598c5791465a9d5ddb2b9977f"}
Oct 30 01:45:18.862326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.862300    4577 scope.go:122] "RemoveContainer" containerID="e5f34ff68b58ac6942d00472fe92e2d1d8c2184598c5791465a9d5ddb2b9977f"
Oct 30 01:45:18.862928 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:18.862897    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm_odh-ai-gateway-infra(88de8ed2-d09a-4715-85f4-35d8a9a51755)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755"
Oct 30 01:45:18.881812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:18.881780    4577 scope.go:122] "RemoveContainer" containerID="ffd9b12611a4c98d5fac27e3bea6d252b1c5a0201ff555379ce703e7aba7ee7c"
Oct 30 01:45:19.018608 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.018525    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"]
Oct 30 01:45:19.068562 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.068526    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:19.082220 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.082178    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:19.099542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.099514    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca\") pod \"dabe84af-0e14-4545-a2a7-1c13a0814000\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") "
Oct 30 01:45:19.099687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.099597    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd\" (UniqueName: \"kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd\") pod \"dabe84af-0e14-4545-a2a7-1c13a0814000\" (UID: \"dabe84af-0e14-4545-a2a7-1c13a0814000\") "
Oct 30 01:45:19.099687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.099638    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794\" (UniqueName: \"kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794\") pod \"f6050dd5-3567-411b-93cb-fe49d4fd7177\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") "
Oct 30 01:45:19.099817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.099765    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca\") pod \"f6050dd5-3567-411b-93cb-fe49d4fd7177\" (UID: \"f6050dd5-3567-411b-93cb-fe49d4fd7177\") "
Oct 30 01:45:19.099915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.099890    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca" pod "dabe84af-0e14-4545-a2a7-1c13a0814000" (UID: "dabe84af-0e14-4545-a2a7-1c13a0814000"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:19.100240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.100209    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca" pod "f6050dd5-3567-411b-93cb-fe49d4fd7177" (UID: "f6050dd5-3567-411b-93cb-fe49d4fd7177"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:19.100640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.100617    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6050dd5-3567-411b-93cb-fe49d4fd7177-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:19.100710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.100646    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabe84af-0e14-4545-a2a7-1c13a0814000-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:19.102162 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.102136    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd" pod "dabe84af-0e14-4545-a2a7-1c13a0814000" (UID: "dabe84af-0e14-4545-a2a7-1c13a0814000"). InnerVolumeSpecName "kube-api-access-ct8bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:19.102423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.102409    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794" pod "f6050dd5-3567-411b-93cb-fe49d4fd7177" (UID: "f6050dd5-3567-411b-93cb-fe49d4fd7177"). InnerVolumeSpecName "kube-api-access-8q794". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:19.104081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.104059    4577 scope.go:122] "RemoveContainer" containerID="41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc"
Oct 30 01:45:19.104178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.104163    4577 scope.go:122] "RemoveContainer" containerID="7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090"
Oct 30 01:45:19.191048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.191003    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"]
Oct 30 01:45:19.202024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.202002    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct8bd\" (UniqueName: \"kubernetes.io/projected/dabe84af-0e14-4545-a2a7-1c13a0814000-kube-api-access-ct8bd\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:19.202024 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.202024    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q794\" (UniqueName: \"kubernetes.io/projected/f6050dd5-3567-411b-93cb-fe49d4fd7177-kube-api-access-8q794\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:19.876997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.876973    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"
Oct 30 01:45:19.876997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.876983    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k" event={"ID":"dabe84af-0e14-4545-a2a7-1c13a0814000","Type":"ContainerDied","Data":"e8bef6cc976c1b29adcdb2d5bcb5a040a6d562ddff80d339c8586b6eae7e9bbb"}
Oct 30 01:45:19.877219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.877069    4577 scope.go:122] "RemoveContainer" containerID="5cdf993420fdd4f18c26175653bec069a92e91cbe76113b7afd156643b6fcb18"
Oct 30 01:45:19.879470 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.879448    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"
Oct 30 01:45:19.879669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.879638    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn" event={"ID":"f6050dd5-3567-411b-93cb-fe49d4fd7177","Type":"ContainerDied","Data":"e3b3b9bd3646fe321a0abec63abb5d3376489ae43f61a0068d91d8af1e5dd949"}
Oct 30 01:45:19.882110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.882087    4577 generic.go:356] "Generic (PLEG): container finished" podID="e45dc766-3e15-4a91-a753-de02ff246993" containerID="3290c1af30c54a256e4c1bc57cbf59f19627adba9b5a1f310cf67bebda3b1274" exitCode=60
Oct 30 01:45:19.882369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.882352    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerDied","Data":"3290c1af30c54a256e4c1bc57cbf59f19627adba9b5a1f310cf67bebda3b1274"}
Oct 30 01:45:19.882957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.882941    4577 scope.go:122] "RemoveContainer" containerID="3290c1af30c54a256e4c1bc57cbf59f19627adba9b5a1f310cf67bebda3b1274"
Oct 30 01:45:19.883413 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:19.883386    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g_odh-ai-gateway-infra(e45dc766-3e15-4a91-a753-de02ff246993)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" podUID="e45dc766-3e15-4a91-a753-de02ff246993"
Oct 30 01:45:19.892391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.892359    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:19.895210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.895191    4577 scope.go:122] "RemoveContainer" containerID="75cf035b92189c13568127093a23dd3647fcf69f24a1327c443274da5f4fd47d"
Oct 30 01:45:19.896464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.895900    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerDied","Data":"3a6c66cfadf7fad33456640a92d6159b498b961e4de3b39983e037029a6b39e9"}
Oct 30 01:45:19.896464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.895835    4577 generic.go:356] "Generic (PLEG): container finished" podID="48323c68-17e2-4c68-9169-f7a61b27411f" containerID="3a6c66cfadf7fad33456640a92d6159b498b961e4de3b39983e037029a6b39e9" exitCode=60
Oct 30 01:45:19.896464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.896353    4577 scope.go:122] "RemoveContainer" containerID="3a6c66cfadf7fad33456640a92d6159b498b961e4de3b39983e037029a6b39e9"
Oct 30 01:45:19.897991 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:19.897551    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb_odh-ai-gateway-infra(48323c68-17e2-4c68-9169-f7a61b27411f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" podUID="48323c68-17e2-4c68-9169-f7a61b27411f"
Oct 30 01:45:19.899565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.899539    4577 generic.go:356] "Generic (PLEG): container finished" podID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerID="045693c670381beaab86d518f13259a5db27bdac3360b1272bdee1605df0e64d" exitCode=60
Oct 30 01:45:19.899669 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.899610    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerDied","Data":"045693c670381beaab86d518f13259a5db27bdac3360b1272bdee1605df0e64d"}
Oct 30 01:45:19.900526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.900083    4577 scope.go:122] "RemoveContainer" containerID="045693c670381beaab86d518f13259a5db27bdac3360b1272bdee1605df0e64d"
Oct 30 01:45:19.900526 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:19.900463    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn_odh-ai-gateway-infra(1baf66d7-a06c-41b0-af3a-47d379b7ae30)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30"
Oct 30 01:45:19.903571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.903547    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerStarted","Data":"3f4821570c424e51181d520aa0b0ca204d47d5b465f54ebde32fef8a995dda9e"}
Oct 30 01:45:19.906911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.906263    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerStarted","Data":"514f9d9cc75b5fb9c34e58f564811ced17e7eb2483bf72551c8e72e51f6ff798"}
Oct 30 01:45:19.908030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.908009    4577 scope.go:122] "RemoveContainer" containerID="2fddd2fafb8da8794e1a8228e1ff45a8b332a247dcaf8af9b499fc34d8cdfd3f"
Oct 30 01:45:19.999141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:19.999111    4577 scope.go:122] "RemoveContainer" containerID="68cdd693b0295c6795114b7f6b112711f042b91adbb8d5199371d8a61a8e89c1"
Oct 30 01:45:20.041294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.041054    4577 scope.go:122] "RemoveContainer" containerID="387b8875330bb7b5336540ced61dd6174d21c15d35485c4eed5e43618e310a7c"
Oct 30 01:45:20.077870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.077759    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"]
Oct 30 01:45:20.083945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.083916    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"]
Oct 30 01:45:20.101317 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.101288    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-0e1a98-29888745-ttn6k"]
Oct 30 01:45:20.136492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.136460    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" path="/var/lib/kubelet/pods/dabe84af-0e14-4545-a2a7-1c13a0814000/volumes"
Oct 30 01:45:20.141993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.141908    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:20.214609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.214578    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"]
Oct 30 01:45:20.256830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.256757    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"]
Oct 30 01:45:20.261414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.261388    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc\" (UniqueName: \"kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc\") pod \"88de8ed2-d09a-4715-85f4-35d8a9a51755\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") "
Oct 30 01:45:20.261723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.261699    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca\") pod \"88de8ed2-d09a-4715-85f4-35d8a9a51755\" (UID: \"88de8ed2-d09a-4715-85f4-35d8a9a51755\") "
Oct 30 01:45:20.262201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.262163    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca" pod "88de8ed2-d09a-4715-85f4-35d8a9a51755" (UID: "88de8ed2-d09a-4715-85f4-35d8a9a51755"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:20.263025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.262714    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88de8ed2-d09a-4715-85f4-35d8a9a51755-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:20.265281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.265255    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc" pod "88de8ed2-d09a-4715-85f4-35d8a9a51755" (UID: "88de8ed2-d09a-4715-85f4-35d8a9a51755"). InnerVolumeSpecName "kube-api-access-579qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:20.285828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.285805    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"]
Oct 30 01:45:20.294675 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.294652    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:20.295308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.295281    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-94ea1e-29888745-ppphn"]
Oct 30 01:45:20.363820 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.363776    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca\") pod \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") "
Oct 30 01:45:20.364245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.364219    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca" pod "0cf87902-7cb2-4647-aca3-33b9b7e65b6a" (UID: "0cf87902-7cb2-4647-aca3-33b9b7e65b6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:20.364379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.364297    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td\" (UniqueName: \"kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td\") pod \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\" (UID: \"0cf87902-7cb2-4647-aca3-33b9b7e65b6a\") "
Oct 30 01:45:20.365305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.365284    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:20.365461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.365438    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-579qc\" (UniqueName: \"kubernetes.io/projected/88de8ed2-d09a-4715-85f4-35d8a9a51755-kube-api-access-579qc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:20.366769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.366745    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td" pod "0cf87902-7cb2-4647-aca3-33b9b7e65b6a" (UID: "0cf87902-7cb2-4647-aca3-33b9b7e65b6a"). InnerVolumeSpecName "kube-api-access-dk4td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:20.467606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.467565    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk4td\" (UniqueName: \"kubernetes.io/projected/0cf87902-7cb2-4647-aca3-33b9b7e65b6a-kube-api-access-dk4td\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:20.924114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.924084    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r" event={"ID":"0cf87902-7cb2-4647-aca3-33b9b7e65b6a","Type":"ContainerDied","Data":"c2ab095a90f02dff3e0c99412a2b65fc5fe84f03ea86649d6b03004aad08ad1d"}
Oct 30 01:45:20.924233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.924170    4577 scope.go:122] "RemoveContainer" containerID="c10591251b26aa364710be5304425507949a5b439b564a841626a095a9447ecd"
Oct 30 01:45:20.924233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.924096    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"
Oct 30 01:45:20.936056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.936024    4577 generic.go:356] "Generic (PLEG): container finished" podID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerID="3f4821570c424e51181d520aa0b0ca204d47d5b465f54ebde32fef8a995dda9e" exitCode=60
Oct 30 01:45:20.936171 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.936155    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerDied","Data":"3f4821570c424e51181d520aa0b0ca204d47d5b465f54ebde32fef8a995dda9e"}
Oct 30 01:45:20.936657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.936636    4577 scope.go:122] "RemoveContainer" containerID="3f4821570c424e51181d520aa0b0ca204d47d5b465f54ebde32fef8a995dda9e"
Oct 30 01:45:20.937214 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:20.937185    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29888745-sg6fp_odh-ai-gateway-infra(a251e7e1-745e-4870-9e8c-0e1c3513d789)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789"
Oct 30 01:45:20.939346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.939323    4577 scope.go:122] "RemoveContainer" containerID="41629cdcd425b455e6cdc547aadc03ca79c11ca06ec2b1f0e45ae8c9b03b8ebc"
Oct 30 01:45:20.941432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.940477    4577 generic.go:356] "Generic (PLEG): container finished" podID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerID="514f9d9cc75b5fb9c34e58f564811ced17e7eb2483bf72551c8e72e51f6ff798" exitCode=60
Oct 30 01:45:20.941432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.940552    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerDied","Data":"514f9d9cc75b5fb9c34e58f564811ced17e7eb2483bf72551c8e72e51f6ff798"}
Oct 30 01:45:20.941432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.941026    4577 scope.go:122] "RemoveContainer" containerID="514f9d9cc75b5fb9c34e58f564811ced17e7eb2483bf72551c8e72e51f6ff798"
Oct 30 01:45:20.941432 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:20.941379    4577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb_odh-ai-gateway-infra(2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c"
Oct 30 01:45:20.946880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.944069    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm" event={"ID":"88de8ed2-d09a-4715-85f4-35d8a9a51755","Type":"ContainerDied","Data":"d25e9bf8f384339bdd13a3d7be65a40590bb7fc541cb7c43bb4d1f57f1e7726a"}
Oct 30 01:45:20.946880 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.944156    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"
Oct 30 01:45:20.974147 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:20.972907    4577 scope.go:122] "RemoveContainer" containerID="7726ff4a8e1e78e82ea798fa400d660492168f479354a3bbea92982cf6878090"
Oct 30 01:45:21.016370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.016342    4577 scope.go:122] "RemoveContainer" containerID="e5f34ff68b58ac6942d00472fe92e2d1d8c2184598c5791465a9d5ddb2b9977f"
Oct 30 01:45:21.094998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.094142    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"]
Oct 30 01:45:21.100081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.099978    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-35042a-29888745-lpg4r"]
Oct 30 01:45:21.157224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.156808    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"]
Oct 30 01:45:21.187902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.187839    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:21.195079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.195050    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca\") pod \"48323c68-17e2-4c68-9169-f7a61b27411f\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") "
Oct 30 01:45:21.195510 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.195474    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca" pod "48323c68-17e2-4c68-9169-f7a61b27411f" (UID: "48323c68-17e2-4c68-9169-f7a61b27411f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:21.195791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.195765    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w\" (UniqueName: \"kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w\") pod \"48323c68-17e2-4c68-9169-f7a61b27411f\" (UID: \"48323c68-17e2-4c68-9169-f7a61b27411f\") "
Oct 30 01:45:21.198429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.198411    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48323c68-17e2-4c68-9169-f7a61b27411f-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:21.200965 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.200942    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w" pod "48323c68-17e2-4c68-9169-f7a61b27411f" (UID: "48323c68-17e2-4c68-9169-f7a61b27411f"). InnerVolumeSpecName "kube-api-access-hrl4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:21.250408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.250383    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:21.256352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.256328    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:21.299624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.299594    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca\") pod \"e45dc766-3e15-4a91-a753-de02ff246993\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") "
Oct 30 01:45:21.299819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.299685    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca\") pod \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") "
Oct 30 01:45:21.299920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.299840    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp\" (UniqueName: \"kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp\") pod \"e45dc766-3e15-4a91-a753-de02ff246993\" (UID: \"e45dc766-3e15-4a91-a753-de02ff246993\") "
Oct 30 01:45:21.300041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.300022    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b\" (UniqueName: \"kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b\") pod \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\" (UID: \"1baf66d7-a06c-41b0-af3a-47d379b7ae30\") "
Oct 30 01:45:21.300108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.300029    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca" pod "e45dc766-3e15-4a91-a753-de02ff246993" (UID: "e45dc766-3e15-4a91-a753-de02ff246993"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:21.300108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.300086    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca" pod "1baf66d7-a06c-41b0-af3a-47d379b7ae30" (UID: "1baf66d7-a06c-41b0-af3a-47d379b7ae30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:21.300600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.300578    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45dc766-3e15-4a91-a753-de02ff246993-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:21.300600 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.300599    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baf66d7-a06c-41b0-af3a-47d379b7ae30-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:21.300748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.300609    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrl4w\" (UniqueName: \"kubernetes.io/projected/48323c68-17e2-4c68-9169-f7a61b27411f-kube-api-access-hrl4w\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:21.301834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.301813    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"]
Oct 30 01:45:21.302228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.302206    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b" pod "1baf66d7-a06c-41b0-af3a-47d379b7ae30" (UID: "1baf66d7-a06c-41b0-af3a-47d379b7ae30"). InnerVolumeSpecName "kube-api-access-pp27b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:21.302412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.302392    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp" pod "e45dc766-3e15-4a91-a753-de02ff246993" (UID: "e45dc766-3e15-4a91-a753-de02ff246993"). InnerVolumeSpecName "kube-api-access-q5dzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:21.318705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.318679    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-499482-29888745-9vjfm"]
Oct 30 01:45:21.402392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.402352    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp27b\" (UniqueName: \"kubernetes.io/projected/1baf66d7-a06c-41b0-af3a-47d379b7ae30-kube-api-access-pp27b\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:21.402392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.402390    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5dzp\" (UniqueName: \"kubernetes.io/projected/e45dc766-3e15-4a91-a753-de02ff246993-kube-api-access-q5dzp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:21.402664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.402540    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"]
Oct 30 01:45:21.812941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.812911    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:21.961194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.961158    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"
Oct 30 01:45:21.961194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.961193    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g" event={"ID":"e45dc766-3e15-4a91-a753-de02ff246993","Type":"ContainerDied","Data":"bc8277980d30cbffa27d6e7685be05073d95ba8d7509f64d600b12517befd911"}
Oct 30 01:45:21.961500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.961258    4577 scope.go:122] "RemoveContainer" containerID="3290c1af30c54a256e4c1bc57cbf59f19627adba9b5a1f310cf67bebda3b1274"
Oct 30 01:45:21.965034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.964588    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"
Oct 30 01:45:21.965034 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.964590    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb" event={"ID":"48323c68-17e2-4c68-9169-f7a61b27411f","Type":"ContainerDied","Data":"5d876f957c5a329e2089f0ec907fd98cc30633a65e61616fa2b68fba6f9768ab"}
Oct 30 01:45:21.968468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.966422    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"
Oct 30 01:45:21.968468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:21.966562    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn" event={"ID":"1baf66d7-a06c-41b0-af3a-47d379b7ae30","Type":"ContainerDied","Data":"af1918515b7b881d94a4f3cdaec4b5c44dcd0f1a21d00b02b370b355a04a26ae"}
Oct 30 01:45:22.002885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.002641    4577 scope.go:122] "RemoveContainer" containerID="3a6c66cfadf7fad33456640a92d6159b498b961e4de3b39983e037029a6b39e9"
Oct 30 01:45:22.030619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.030585    4577 scope.go:122] "RemoveContainer" containerID="045693c670381beaab86d518f13259a5db27bdac3360b1272bdee1605df0e64d"
Oct 30 01:45:22.120163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.120140    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" path="/var/lib/kubelet/pods/0cf87902-7cb2-4647-aca3-33b9b7e65b6a/volumes"
Oct 30 01:45:22.120757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.120742    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" path="/var/lib/kubelet/pods/88de8ed2-d09a-4715-85f4-35d8a9a51755/volumes"
Oct 30 01:45:22.121514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.121499    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" path="/var/lib/kubelet/pods/f6050dd5-3567-411b-93cb-fe49d4fd7177/volumes"
Oct 30 01:45:22.196559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.196528    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"]
Oct 30 01:45:22.239150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.239113    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-bc127f-29888745-5tqjb"]
Oct 30 01:45:22.264005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.261574    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"]
Oct 30 01:45:22.266890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.266836    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:22.268907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.268885    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:22.269222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.269201    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-9da49f-29888745-c9s6g"]
Oct 30 01:45:22.298538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.298509    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"]
Oct 30 01:45:22.302566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.302544    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-1854c4-29888745-4mdxn"]
Oct 30 01:45:22.319468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.319446    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p\" (UniqueName: \"kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p\") pod \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") "
Oct 30 01:45:22.319582 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.319549    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf\" (UniqueName: \"kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf\") pod \"a251e7e1-745e-4870-9e8c-0e1c3513d789\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") "
Oct 30 01:45:22.319771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.319754    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca\") pod \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\" (UID: \"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c\") "
Oct 30 01:45:22.319907 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.319885    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca\") pod \"a251e7e1-745e-4870-9e8c-0e1c3513d789\" (UID: \"a251e7e1-745e-4870-9e8c-0e1c3513d789\") "
Oct 30 01:45:22.320170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.320104    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca" pod "2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" (UID: "2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:22.320170 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.320180    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca" pod "a251e7e1-745e-4870-9e8c-0e1c3513d789" (UID: "a251e7e1-745e-4870-9e8c-0e1c3513d789"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:22.320539 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.320521    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:22.320614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.320541    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a251e7e1-745e-4870-9e8c-0e1c3513d789-maas-api-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:22.321984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.321925    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf" pod "a251e7e1-745e-4870-9e8c-0e1c3513d789" (UID: "a251e7e1-745e-4870-9e8c-0e1c3513d789"). InnerVolumeSpecName "kube-api-access-6wlqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:22.321984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.321964    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p" pod "2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" (UID: "2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c"). InnerVolumeSpecName "kube-api-access-hsk9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:22.422832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.422797    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsk9p\" (UniqueName: \"kubernetes.io/projected/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c-kube-api-access-hsk9p\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:22.423114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:22.423094    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wlqf\" (UniqueName: \"kubernetes.io/projected/a251e7e1-745e-4870-9e8c-0e1c3513d789-kube-api-access-6wlqf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:23.015413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.015382    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"
Oct 30 01:45:23.015596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.015414    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp" event={"ID":"a251e7e1-745e-4870-9e8c-0e1c3513d789","Type":"ContainerDied","Data":"a4776ee5bd51cdde29245b8a8010f1ceab3e3735a51855b4b5887784775a49c1"}
Oct 30 01:45:23.015596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.015504    4577 scope.go:122] "RemoveContainer" containerID="3f4821570c424e51181d520aa0b0ca204d47d5b465f54ebde32fef8a995dda9e"
Oct 30 01:45:23.020711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.020422    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"
Oct 30 01:45:23.020711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.020452    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb" event={"ID":"2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c","Type":"ContainerDied","Data":"de80a049dcf27d0d8c4cd357c25cc78e9fd0a1381ad5a1d8c3aa7102d081853a"}
Oct 30 01:45:23.031055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.031035    4577 scope.go:122] "RemoveContainer" containerID="514f9d9cc75b5fb9c34e58f564811ced17e7eb2483bf72551c8e72e51f6ff798"
Oct 30 01:45:23.080412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.080385    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"]
Oct 30 01:45:23.093890 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.093863    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888745-sg6fp"]
Oct 30 01:45:23.122143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.122119    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"]
Oct 30 01:45:23.126143 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.126122    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-bdf6ca-29888745-s4tdb"]
Oct 30 01:45:23.624055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:23.624011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:24.114244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:24.114212    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" path="/var/lib/kubelet/pods/1baf66d7-a06c-41b0-af3a-47d379b7ae30/volumes"
Oct 30 01:45:24.114715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:24.114701    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" path="/var/lib/kubelet/pods/2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c/volumes"
Oct 30 01:45:24.115163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:24.115150    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" path="/var/lib/kubelet/pods/48323c68-17e2-4c68-9169-f7a61b27411f/volumes"
Oct 30 01:45:24.115881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:24.115867    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" path="/var/lib/kubelet/pods/a251e7e1-745e-4870-9e8c-0e1c3513d789/volumes"
Oct 30 01:45:24.116276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:24.116263    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e45dc766-3e15-4a91-a753-de02ff246993" path="/var/lib/kubelet/pods/e45dc766-3e15-4a91-a753-de02ff246993/volumes"
Oct 30 01:45:25.387091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:25.387058    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:26.212874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:26.212811    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:27.022357 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:27.022308    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:45:27.150571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:27.150534    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:28.730522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:28.730483    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" podUID="c340ad11-f797-4128-abea-4ee3c7181708" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:45:29.003239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.003152    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:29.411441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.411402    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:45:29.569626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.569532    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume\") pod \"c340ad11-f797-4128-abea-4ee3c7181708\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") "
Oct 30 01:45:29.569814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.569631    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7\" (UniqueName: \"kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7\") pod \"c340ad11-f797-4128-abea-4ee3c7181708\" (UID: \"c340ad11-f797-4128-abea-4ee3c7181708\") "
Oct 30 01:45:29.570011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.569984    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume" pod "c340ad11-f797-4128-abea-4ee3c7181708" (UID: "c340ad11-f797-4128-abea-4ee3c7181708"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:45:29.570814 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.570785    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c340ad11-f797-4128-abea-4ee3c7181708-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:29.572227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.572199    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7" pod "c340ad11-f797-4128-abea-4ee3c7181708" (UID: "c340ad11-f797-4128-abea-4ee3c7181708"). InnerVolumeSpecName "kube-api-access-xqfh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:45:29.672696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:29.672665    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqfh7\" (UniqueName: \"kubernetes.io/projected/c340ad11-f797-4128-abea-4ee3c7181708-kube-api-access-xqfh7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:45:30.113879 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.113819    4577 generic.go:356] "Generic (PLEG): container finished" podID="c340ad11-f797-4128-abea-4ee3c7181708" containerID="d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f" exitCode=0
Oct 30 01:45:30.114286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.113921    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"
Oct 30 01:45:30.115049 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.115026    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" event={"ID":"c340ad11-f797-4128-abea-4ee3c7181708","Type":"ContainerDied","Data":"d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f"}
Oct 30 01:45:30.115126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.115077    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c" event={"ID":"c340ad11-f797-4128-abea-4ee3c7181708","Type":"ContainerDied","Data":"483c26b5cb8c9c20cb9d570207dbe15d0b532a04848e33e46907fde8e1863d6b"}
Oct 30 01:45:30.115126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.115107    4577 scope.go:122] "RemoveContainer" containerID="d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f"
Oct 30 01:45:30.128029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.128006    4577 scope.go:122] "RemoveContainer" containerID="d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f"
Oct 30 01:45:30.128349 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:30.128323    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f\": container with ID starting with d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f not found: ID does not exist" containerID="d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f"
Oct 30 01:45:30.128439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.128360    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f"} err="failed to get container status \"d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f\": rpc error: code = NotFound desc = could not find container \"d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f\": container with ID starting with d9f2c25cfc05a90c1f8e84d7f4db76f1e7ec2aa851b4ba3642c20456cda14a2f not found: ID does not exist"
Oct 30 01:45:30.176104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.176065    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"]
Oct 30 01:45:30.198075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.198047    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5457f2-589dd45787-rmz5c"]
Oct 30 01:45:30.752664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:30.752626    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:32.113639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:32.113611    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c340ad11-f797-4128-abea-4ee3c7181708" path="/var/lib/kubelet/pods/c340ad11-f797-4128-abea-4ee3c7181708/volumes"
Oct 30 01:45:32.398508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:32.398423    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:34.297391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:34.297359    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:35.976255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:35.976220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:37.833120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:37.833087    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:39.705694 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:39.705662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:40.464298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:40.464260    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:40.584011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:40.583980    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:41.389902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:41.389825    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:43.127002 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:43.126962    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:44.819964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:44.819930    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:46.284507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:46.284478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:46.809863 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:46.809814    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:46.929387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:46.929348    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:47.524637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:47.524602    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:48.700810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:48.700766    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:49.273865 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:49.269497    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:49.413422 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:49.413386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:49.549303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:49.549210    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:49.953107 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:49.952989    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:51.738549 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:51.738516    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:53.381224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:53.381194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:53.494500 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:53.494472    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:53.612872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:53.610720    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:53.732925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:53.732809    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:54.635174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:54.635138    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:54.758089 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:54.758060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:55.349341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:55.349293    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:56.942373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:56.942327    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:57.015360 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:45:57.015324    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:45:57.078640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:57.078595    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:57.240722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:57.240686    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"]
Oct 30 01:45:57.241010 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:57.240986    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy" containerID="cri-o://59d08c1c96f2f924316aca4e4286cb0822a4c5b81a563253240a7a2b392b1303" gracePeriod=30
Oct 30 01:45:57.243682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:57.243650    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:57.363522 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:57.363488    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:58.182148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:58.182110    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:58.307135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:58.307093    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:59.108190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:59.108126    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:59.228962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:59.228928    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:59.346210 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:59.346168    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:45:59.475374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:45:59.475285    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:00.917418 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:00.917379    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:02.478878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.477498    4577 generic.go:356] "Generic (PLEG): container finished" podID="2687b657-b370-4dc5-bf86-516d3111160f" containerID="59d08c1c96f2f924316aca4e4286cb0822a4c5b81a563253240a7a2b392b1303" exitCode=0
Oct 30 01:46:02.478878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.477528    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" event={"ID":"2687b657-b370-4dc5-bf86-516d3111160f","Type":"ContainerDied","Data":"59d08c1c96f2f924316aca4e4286cb0822a4c5b81a563253240a7a2b392b1303"}
Oct 30 01:46:02.478878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.477601    4577 scope.go:122] "RemoveContainer" containerID="3d82c9a666a03b4e15bfe3badae79b02f052de1289684c7ccdeac8775ddb08da"
Oct 30 01:46:02.560446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.560413    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:46:02.588054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.588016    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.588778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.588631    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.589451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.588785    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.591008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.590749    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.591700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.590136    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:02.591813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.590495    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:02.591813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.590544    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:02.591813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.591790    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.592004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.591867    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.594400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.594373    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:46:02.594534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.594436    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.594534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.594492    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.594534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.594521    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token\") pod \"2687b657-b370-4dc5-bf86-516d3111160f\" (UID: \"2687b657-b370-4dc5-bf86-516d3111160f\") "
Oct 30 01:46:02.596174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.596054    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:02.596448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.596423    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.596553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.596454    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2687b657-b370-4dc5-bf86-516d3111160f-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.596553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.596468    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.596553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.596480    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.596553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.596492    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.598368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.598340    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:02.599238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.599213    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "kube-api-access-t5pvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:46:02.600214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.600174    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:46:02.600426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.600406    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo" pod "2687b657-b370-4dc5-bf86-516d3111160f" (UID: "2687b657-b370-4dc5-bf86-516d3111160f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:46:02.669356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.669307    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:02.699045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.699008    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2687b657-b370-4dc5-bf86-516d3111160f-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.699045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.699043    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5pvg\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-kube-api-access-t5pvg\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.699382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.699060    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2687b657-b370-4dc5-bf86-516d3111160f-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:02.699382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:02.699077    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2687b657-b370-4dc5-bf86-516d3111160f-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:03.489323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:03.489290    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"
Oct 30 01:46:03.489323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:03.489302    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4" event={"ID":"2687b657-b370-4dc5-bf86-516d3111160f","Type":"ContainerDied","Data":"4c4d006c15e4c479017abe7803c9fb53779b1c75d8ba78e16b9730813008df06"}
Oct 30 01:46:03.489888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:03.489389    4577 scope.go:122] "RemoveContainer" containerID="59d08c1c96f2f924316aca4e4286cb0822a4c5b81a563253240a7a2b392b1303"
Oct 30 01:46:03.563864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:03.563818    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"]
Oct 30 01:46:03.578032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:03.577992    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5457f2-openshift-default-69884bcb8c-b45z4"]
Oct 30 01:46:04.113204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:04.113176    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2687b657-b370-4dc5-bf86-516d3111160f" path="/var/lib/kubelet/pods/2687b657-b370-4dc5-bf86-516d3111160f/volumes"
Oct 30 01:46:04.426159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:04.426040    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:05.579133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:05.579100    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:06.189532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:06.189499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:06.532016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:06.531981    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:07.936367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:07.936326    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:08.559240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:08.559206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:09.021174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:09.021141    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:09.274638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:09.274504    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:09.938789 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:09.938748    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:11.321601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:11.321547    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:11.849104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:11.849067    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:11.962617 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:11.962578    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:12.461829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:12.461799    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:12.642897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:12.642862    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:13.581715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:13.581668    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:13.814184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:13.814152    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:14.444722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:14.444687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:14.840872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:14.833002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:16.238729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:16.238693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:18.077211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:18.077161    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-69fm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.1.10:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:46:18.086626 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:18.086586    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.10:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:46:18.358926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:18.358419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:18.470875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:18.469387    4577 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-fcb75c45f-x8q6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.1.9:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:46:18.470875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:18.469447    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.1.9:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:46:18.841372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:18.841324    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:19.033781 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:19.033746    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"]
Oct 30 01:46:19.047606 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:19.047576    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-31383514-9t4zt"]
Oct 30 01:46:20.110135 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:20.110094    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:20.114875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:20.114820    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f618bbf-46e0-4a73-8f29-23c76f1928f0" path="/var/lib/kubelet/pods/6f618bbf-46e0-4a73-8f29-23c76f1928f0/volumes"
Oct 30 01:46:22.043723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:22.043690    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:22.174086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:22.174051    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:22.288897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:22.288865    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:23.830634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:23.830605    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:25.693922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:25.693798    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:26.370570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:26.370535    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:27.015938 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:46:27.015898    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:46:27.475149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:27.475063    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:29.552871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:29.544447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:30.517721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:30.517687    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:31.355074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:31.355045    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:31.985544 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:31.985499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:32.121561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:32.121527    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:32.961342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:32.961305    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:34.760687 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:34.760656    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:36.930019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:36.929979    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:37.082838 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:37.082801    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:38.917051 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:38.917012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:41.433682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:41.433639    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:41.734923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:41.732769    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:41.889511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:41.889478    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:42.113620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:42.113538    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"]
Oct 30 01:46:42.113786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:42.113732    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy" containerID="cri-o://b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833" gracePeriod=30
Oct 30 01:46:42.157673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:42.157643    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:43.792386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:43.792297    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:45.935706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:45.935671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:47.431674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.431647    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:46:47.449374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.449345    4577 generic.go:356] "Generic (PLEG): container finished" podID="b78892cf-9422-4937-b888-0e7d5326fd13" containerID="b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833" exitCode=0
Oct 30 01:46:47.449529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.449477    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" event={"ID":"b78892cf-9422-4937-b888-0e7d5326fd13","Type":"ContainerDied","Data":"b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833"}
Oct 30 01:46:47.449529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.449515    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz" event={"ID":"b78892cf-9422-4937-b888-0e7d5326fd13","Type":"ContainerDied","Data":"c9c577580a900a2da7f1a78a4765879d6ec453f36c46e5a68bf4f8204dbc7d7b"}
Oct 30 01:46:47.449662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.449536    4577 scope.go:122] "RemoveContainer" containerID="b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833"
Oct 30 01:46:47.449662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.449657    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"
Oct 30 01:46:47.466218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.466188    4577 scope.go:122] "RemoveContainer" containerID="60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"
Oct 30 01:46:47.479871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.479799    4577 scope.go:122] "RemoveContainer" containerID="b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833"
Oct 30 01:46:47.480199 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:46:47.480171    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833\": container with ID starting with b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833 not found: ID does not exist" containerID="b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833"
Oct 30 01:46:47.480301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.480212    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833"} err="failed to get container status \"b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833\": rpc error: code = NotFound desc = could not find container \"b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833\": container with ID starting with b75951f830833e8c3ace3677dd5b0624a596ea0a5ce3f2f6c4a81805eeca3833 not found: ID does not exist"
Oct 30 01:46:47.480301 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.480241    4577 scope.go:122] "RemoveContainer" containerID="60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"
Oct 30 01:46:47.480615 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:46:47.480586    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37\": container with ID starting with 60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37 not found: ID does not exist" containerID="60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"
Oct 30 01:46:47.480699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.480622    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37"} err="failed to get container status \"60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37\": rpc error: code = NotFound desc = could not find container \"60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37\": container with ID starting with 60881115aefc4166f9ad4182b5f9d2f5de236f80b54618c7cb3d9292064fbb37 not found: ID does not exist"
Oct 30 01:46:47.565038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565005    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565107    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565152    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565194    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565289    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565340    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565392    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565449    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.565695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.565497    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data\") pod \"b78892cf-9422-4937-b888-0e7d5326fd13\" (UID: \"b78892cf-9422-4937-b888-0e7d5326fd13\") "
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566041    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566218    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566414    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566449    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566940    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566963    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566979    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b78892cf-9422-4937-b888-0e7d5326fd13-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.567057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.566994    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.567711 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.567683    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:47.568091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.568068    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:46:47.568214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.568193    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:46:47.568681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.568663    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:46:47.569499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.569482    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb" pod "b78892cf-9422-4937-b888-0e7d5326fd13" (UID: "b78892cf-9422-4937-b888-0e7d5326fd13"). InnerVolumeSpecName "kube-api-access-xqxcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:46:47.668829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.668754    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b78892cf-9422-4937-b888-0e7d5326fd13-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.668829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.668804    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqxcb\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-kube-api-access-xqxcb\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.668829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.668819    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.668829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.668833    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b78892cf-9422-4937-b888-0e7d5326fd13-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.669105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.668861    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b78892cf-9422-4937-b888-0e7d5326fd13-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:47.811151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.811117    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"]
Oct 30 01:46:47.828129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.828098    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6aa9aa-openshift-default-8d7c8d5d9-67lcz"]
Oct 30 01:46:47.967370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:47.967289    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:48.114671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:48.114641    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" path="/var/lib/kubelet/pods/b78892cf-9422-4937-b888-0e7d5326fd13/volumes"
Oct 30 01:46:49.804825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:49.804793    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:51.609195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:51.609150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:53.284868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:53.284387    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:53.405911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:53.405874    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:53.408027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:53.407994    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"]
Oct 30 01:46:53.467961 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:53.467920    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"]
Oct 30 01:46:53.574657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:53.574455    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:53.759455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:53.749612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:55.642993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:55.642945    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:57.016861 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:46:57.016807    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:46:57.430189 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:57.430091    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:46:58.409190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.409152    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9" podUID="a1761c8f-2e49-4669-ba3f-67d431adfb99" containerName="payload-processing" containerID="cri-o://a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa" gracePeriod=25
Oct 30 01:46:58.469146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.469113    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerName="payload-pre-processing" containerID="cri-o://652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f" gracePeriod=25
Oct 30 01:46:58.698831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.698804    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:46:58.766212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.766177    4577 generic.go:356] "Generic (PLEG): container finished" podID="a1761c8f-2e49-4669-ba3f-67d431adfb99" containerID="a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa" exitCode=0
Oct 30 01:46:58.766413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.766240    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"
Oct 30 01:46:58.766413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.766347    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9" event={"ID":"a1761c8f-2e49-4669-ba3f-67d431adfb99","Type":"ContainerDied","Data":"a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa"}
Oct 30 01:46:58.766413 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.766392    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9" event={"ID":"a1761c8f-2e49-4669-ba3f-67d431adfb99","Type":"ContainerDied","Data":"5acca6e0e0640beb30dff1eb07073db6524757511acf2bc899007a624a9862a8"}
Oct 30 01:46:58.766563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.766422    4577 scope.go:122] "RemoveContainer" containerID="a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa"
Oct 30 01:46:58.780073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.780052    4577 scope.go:122] "RemoveContainer" containerID="a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa"
Oct 30 01:46:58.780539 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:46:58.780517    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa\": container with ID starting with a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa not found: ID does not exist" containerID="a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa"
Oct 30 01:46:58.780654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.780549    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa"} err="failed to get container status \"a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa\": rpc error: code = NotFound desc = could not find container \"a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa\": container with ID starting with a937acf4f27bc11b36f2f5bf1f70e5803f638f387f75a727045f1813505743fa not found: ID does not exist"
Oct 30 01:46:58.783963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.783939    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume\") pod \"a1761c8f-2e49-4669-ba3f-67d431adfb99\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") "
Oct 30 01:46:58.784087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.783995    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b\" (UniqueName: \"kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b\") pod \"a1761c8f-2e49-4669-ba3f-67d431adfb99\" (UID: \"a1761c8f-2e49-4669-ba3f-67d431adfb99\") "
Oct 30 01:46:58.784410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.784344    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume" pod "a1761c8f-2e49-4669-ba3f-67d431adfb99" (UID: "a1761c8f-2e49-4669-ba3f-67d431adfb99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:46:58.784734 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.784714    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1761c8f-2e49-4669-ba3f-67d431adfb99-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:58.787128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.787104    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b" pod "a1761c8f-2e49-4669-ba3f-67d431adfb99" (UID: "a1761c8f-2e49-4669-ba3f-67d431adfb99"). InnerVolumeSpecName "kube-api-access-skh4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:46:58.885773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:58.885742    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skh4b\" (UniqueName: \"kubernetes.io/projected/a1761c8f-2e49-4669-ba3f-67d431adfb99-kube-api-access-skh4b\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:46:59.118048 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:59.118010    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"]
Oct 30 01:46:59.135935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:59.135897    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1a4b17-65fc9479cd-84jx9"]
Oct 30 01:46:59.200802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:46:59.200765    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:00.112446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:00.112416    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1761c8f-2e49-4669-ba3f-67d431adfb99" path="/var/lib/kubelet/pods/a1761c8f-2e49-4669-ba3f-67d431adfb99/volumes"
Oct 30 01:47:00.371940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:00.371826    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:00.982704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:00.982675    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:01.785875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:01.779053    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:03.648583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:03.648550    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:03.906153 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:03.906075    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:47:05.497666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:05.497633    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:07.523217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:07.523181    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:07.651810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:07.651768    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:07.772412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:07.772384    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:09.673464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:09.673432    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:11.823882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:11.823812    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:13.883692 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:13.883655    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:13.937970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:13.937929    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:47:14.045651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:14.045621    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:15.877222 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:15.877177    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:17.797992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:17.797960    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:18.773731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.773705    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:47:18.838875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.837824    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume\") pod \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") "
Oct 30 01:47:18.838875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.838044    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp\" (UniqueName: \"kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp\") pod \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\" (UID: \"5eccd8f4-0d45-4a44-841c-a2b03a4bda04\") "
Oct 30 01:47:18.842078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.842044    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp" pod "5eccd8f4-0d45-4a44-841c-a2b03a4bda04" (UID: "5eccd8f4-0d45-4a44-841c-a2b03a4bda04"). InnerVolumeSpecName "kube-api-access-pxrmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:47:18.842393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.842374    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume" pod "5eccd8f4-0d45-4a44-841c-a2b03a4bda04" (UID: "5eccd8f4-0d45-4a44-841c-a2b03a4bda04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:47:18.940507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.940476    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:18.940507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.940505    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxrmp\" (UniqueName: \"kubernetes.io/projected/5eccd8f4-0d45-4a44-841c-a2b03a4bda04-kube-api-access-pxrmp\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:18.975019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.974978    4577 generic.go:356] "Generic (PLEG): container finished" podID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerID="652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f" exitCode=0
Oct 30 01:47:18.975204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.975043    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"
Oct 30 01:47:18.975204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.975068    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" event={"ID":"5eccd8f4-0d45-4a44-841c-a2b03a4bda04","Type":"ContainerDied","Data":"652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f"}
Oct 30 01:47:18.975204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.975136    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46" event={"ID":"5eccd8f4-0d45-4a44-841c-a2b03a4bda04","Type":"ContainerDied","Data":"e612fbe3d45db81a20764f79eedadee8834b69c9c92cb87b8d752e73c571b7f6"}
Oct 30 01:47:18.975204 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.975167    4577 scope.go:122] "RemoveContainer" containerID="652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f"
Oct 30 01:47:18.987619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.987598    4577 scope.go:122] "RemoveContainer" containerID="652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f"
Oct 30 01:47:18.987928 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:47:18.987904    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f\": container with ID starting with 652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f not found: ID does not exist" containerID="652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f"
Oct 30 01:47:18.988026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:18.987938    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f"} err="failed to get container status \"652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f\": rpc error: code = NotFound desc = could not find container \"652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f\": container with ID starting with 652f836c26cc73e28d6319bee6d2497561b8c12a9c0486edeca99b2f357cc70f not found: ID does not exist"
Oct 30 01:47:19.038044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:19.038011    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"]
Oct 30 01:47:19.043072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:19.043045    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1a4b17-69d86f7847-d7q46"]
Oct 30 01:47:19.805302 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:19.805250    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:20.112622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:20.112549    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" path="/var/lib/kubelet/pods/5eccd8f4-0d45-4a44-841c-a2b03a4bda04/volumes"
Oct 30 01:47:21.531622 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:21.531590    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:23.412208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:23.412176    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:25.226565 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:25.226529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:26.443102 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:26.443065    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:27.009934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:27.009896    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:27.016575 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:47:27.016541    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:47:28.942505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:28.942466    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:30.599805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:30.599768    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:31.146873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:31.136279    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:31.321742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:31.321692    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:31.773620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:31.773586    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:33.417543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:33.417496    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:35.334796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:35.334759    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:36.952840 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:36.952799    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:37.152517 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:37.152482    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:37.306679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:37.306637    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:37.586421 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:37.586334    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:39.686124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:39.686086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:39.804615 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:39.804583    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:40.142498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:40.142462    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"]
Oct 30 01:47:40.142745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:40.142718    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" containerID="cri-o://b3c830012cf12fdbcd4d48fd6a05ca03db40166c2b784aa0993aeb0827161e87" gracePeriod=30
Oct 30 01:47:40.230309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:40.230272    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:40.412085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:40.411995    4577 patch_prober.go:29] interesting pod/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:47:40.412347 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:40.412321    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:47:41.131303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:41.131269    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:41.274456 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:41.274407    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:41.409956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:41.409829    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:41.700591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:41.700502    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:41.946176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:41.946139    4577 scope.go:122] "RemoveContainer" containerID="0ea3d5db2297480d74a9b8a168a9bc41b55653e97620b79e318e83acc1510c78"
Oct 30 01:47:41.957650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:41.957628    4577 scope.go:122] "RemoveContainer" containerID="fcae50843c889879b2b6047a0d21a9601abc92aed898f4e20870396302ae045e"
Oct 30 01:47:43.742033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:43.742002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:45.411146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.411066    4577 generic.go:356] "Generic (PLEG): container finished" podID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerID="b3c830012cf12fdbcd4d48fd6a05ca03db40166c2b784aa0993aeb0827161e87" exitCode=0
Oct 30 01:47:45.411578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.411139    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" event={"ID":"c2e195d5-f3d3-4e96-a1bc-6c898c909c63","Type":"ContainerDied","Data":"b3c830012cf12fdbcd4d48fd6a05ca03db40166c2b784aa0993aeb0827161e87"}
Oct 30 01:47:45.411578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.411200    4577 scope.go:122] "RemoveContainer" containerID="63198f222a246ab9a87b5a8f87506b37178a0ad38e5db18046bc9595ffe5e68d"
Oct 30 01:47:45.461026 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.460992    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:47:45.530227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.530196    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:45.599875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.592489    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600202    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600310    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600502    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600577    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600615    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600727    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600875    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.637891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.600928    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert\") pod \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\" (UID: \"c2e195d5-f3d3-4e96-a1bc-6c898c909c63\") "
Oct 30 01:47:45.638443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.638348    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:47:45.641269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.638507    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:47:45.641269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.638629    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:47:45.641269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.638751    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:47:45.643524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.641653    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "kube-api-access-4tc89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:47:45.643524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.641896    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:47:45.643524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.642443    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:47:45.644762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.644733    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:47:45.647281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.647255    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo" pod "c2e195d5-f3d3-4e96-a1bc-6c898c909c63" (UID: "c2e195d5-f3d3-4e96-a1bc-6c898c909c63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:47:45.738964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.738929    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.738964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.738962    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.738964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.738971    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.739313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.738983    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.739313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.738993    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.739313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.739001    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.739313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.739013    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.739313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.739043    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tc89\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-kube-api-access-4tc89\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:45.739313 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:45.739052    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e195d5-f3d3-4e96-a1bc-6c898c909c63-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:47:46.054931 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.054782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:46.168638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.168593    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:46.420006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.419930    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"
Oct 30 01:47:46.420512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.419931    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k" event={"ID":"c2e195d5-f3d3-4e96-a1bc-6c898c909c63","Type":"ContainerDied","Data":"105962dce5a97169ff93429015be57f446c29f30300abdab8add89fbe639f1d1"}
Oct 30 01:47:46.420512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.420073    4577 scope.go:122] "RemoveContainer" containerID="b3c830012cf12fdbcd4d48fd6a05ca03db40166c2b784aa0993aeb0827161e87"
Oct 30 01:47:46.503221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.503188    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"]
Oct 30 01:47:46.516568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:46.516538    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1a4b17-openshift-default-655494677-wq82k"]
Oct 30 01:47:47.334745 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:47.334710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:47.492944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:47.492897    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:48.156586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:48.156545    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" path="/var/lib/kubelet/pods/c2e195d5-f3d3-4e96-a1bc-6c898c909c63/volumes"
Oct 30 01:47:48.717993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:48.717960    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:48.838062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:48.838027    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:49.294714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:49.294683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:51.112025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:51.111993    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:52.748149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:52.748119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:53.033196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:53.033121    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:54.716970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:54.716927    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:56.531046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:56.531012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:47:57.015238 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:47:57.015195    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:47:58.407702 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:47:58.407665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:00.166995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:00.166959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:00.292225 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:00.292189    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:02.135731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:02.135701    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:02.826691 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:02.826651    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:02.995227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:02.995192    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:04.042515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:04.042479    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:04.997286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:04.997249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:05.833636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:05.833600    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:05.956050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:05.956012    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:06.481683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:06.481636    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:07.845613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:07.845580    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:09.906062 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:09.906008    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:11.161069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:11.161035    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:11.806874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:11.806829    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:12.923581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:12.923503    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:13.639251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:13.639223    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:15.318310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:15.318274    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:16.387908 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:16.387868    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:17.229627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:17.229579    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:18.962365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:18.962323    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:20.704764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:20.704732    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:22.612727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:22.612694    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:24.303448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:24.303410    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:26.038887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:26.038835    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:26.557701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:26.557665    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:27.015012 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:48:27.014970    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:48:27.984824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:27.984780    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:29.708007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:29.707959    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:31.465894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:31.465820    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:33.142425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:33.142376    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:34.917579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:34.917546    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:36.623282 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:36.623247    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:38.468992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:38.468958    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:40.202560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:40.202529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:40.828780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:40.828743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:42.185883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:42.183928    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:43.915769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:43.915739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:45.610021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:45.609991    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:47.377710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:47.377660    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:49.116367 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:49.116313    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:51.095155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:51.095118    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:52.792254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:52.792221    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:53.351092 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:53.351052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:54.598547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:54.598512    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:56.308055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:56.308024    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:57.014172 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:48:57.014137    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:48:57.965357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:57.965307    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:48:59.676330 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:48:59.676296    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:01.439559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:01.439526    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:03.211439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:03.211406    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:05.102830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:05.102797    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:06.981370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:06.981328    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:08.788257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:08.788195    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:10.486425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:10.486381    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:12.305862 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.305821    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:12.717868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.717764    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"]
Oct 30 01:49:12.718957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.718930    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.718957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.718955    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.718974    4577 cpu_manager.go:414] "containerMap: removing container" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.718982    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.718994    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719002    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719014    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719022    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719030    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719037    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719047    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719054    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719064    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719071    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719086    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719092    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719105    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719112    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719122    4577 cpu_manager.go:414] "containerMap: removing container" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719129    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719148    4577 cpu_manager.go:414] "containerMap: removing container" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719155    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719174    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719181    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy"
Oct 30 01:49:12.719183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719192    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719198    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719209    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719216    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719226    4577 cpu_manager.go:414] "containerMap: removing container" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719233    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719245    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719253    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719276    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719283    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719294    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719300    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719310    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719317    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719326    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719333    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719346    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719352    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719373    4577 cpu_manager.go:414] "containerMap: removing container" podUID="a1761c8f-2e49-4669-ba3f-67d431adfb99" containerName="payload-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719381    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1761c8f-2e49-4669-ba3f-67d431adfb99" containerName="payload-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719396    4577 cpu_manager.go:414] "containerMap: removing container" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719403    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719413    4577 cpu_manager.go:414] "containerMap: removing container" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719420    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719429    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719435    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719450    4577 cpu_manager.go:414] "containerMap: removing container" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719457    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719466    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerName="payload-pre-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719473    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerName="payload-pre-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719484    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c340ad11-f797-4128-abea-4ee3c7181708" containerName="payload-pre-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719491    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c340ad11-f797-4128-abea-4ee3c7181708" containerName="payload-pre-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719516    4577 cpu_manager.go:414] "containerMap: removing container" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719523    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719682    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719698    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719707    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719716    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719725    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719735    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1761c8f-2e49-4669-ba3f-67d431adfb99" containerName="payload-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719745    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719754    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719761    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719769    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719777    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719784    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eccd8f4-0d45-4a44-841c-a2b03a4bda04" containerName="payload-pre-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719794    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719802    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c340ad11-f797-4128-abea-4ee3c7181708" containerName="payload-pre-processing"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719812    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719821    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719832    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719858    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719869    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719878    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719885    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719893    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719902    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719912    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719923    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719935    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719949    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719957    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="a251e7e1-745e-4870-9e8c-0e1c3513d789" containerName="cleanup"
Oct 30 01:49:12.720484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.719965    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="b78892cf-9422-4937-b888-0e7d5326fd13" containerName="istio-proxy"
Oct 30 01:49:12.724541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.724517    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.742519 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.742486    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"]
Oct 30 01:49:12.747893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.747709    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"default-dockercfg-ndxl7\""
Oct 30 01:49:12.747893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.747797    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"openshift-service-ca.crt\""
Oct 30 01:49:12.758896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.754755    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"rate-test-model-9da49f-kserve-self-signed-certs\""
Oct 30 01:49:12.758896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.754836    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-9da49f\"/\"kube-root-ca.crt\""
Oct 30 01:49:12.767826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.767791    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:12.815308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.815276    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.815461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.815413    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.815512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.815478    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.815512 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.815494    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.815624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.815603    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.815741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.815723    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfc6f\" (UniqueName: \"kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917079    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917167    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfc6f\" (UniqueName: \"kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917263    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917351    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917397    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917414    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917540    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917614 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917598    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.917881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.917829    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.919905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.919873    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.920142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.920121    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:12.988871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:12.986587    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfc6f\" (UniqueName: \"kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f\") pod \"rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") " pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:13.056645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:13.056573    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:13.423443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:13.423415    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"]
Oct 30 01:49:13.424182 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:49:13.424153    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8c1a9a9_9b24_446b_b543_99e293cece15.slice/crio-e2fb73a9f703a8624e50ba648bcc06c744030a1bbc967620f8e7fc96f689b2b6 WatchSource:0}: Error finding container e2fb73a9f703a8624e50ba648bcc06c744030a1bbc967620f8e7fc96f689b2b6: Status 404 returned error can't find the container with id e2fb73a9f703a8624e50ba648bcc06c744030a1bbc967620f8e7fc96f689b2b6
Oct 30 01:49:14.092534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:14.092499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:14.275824 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:14.275785    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" event={"ID":"f8c1a9a9-9b24-446b-b543-99e293cece15","Type":"ContainerStarted","Data":"d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c"}
Oct 30 01:49:14.276023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:14.275835    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" event={"ID":"f8c1a9a9-9b24-446b-b543-99e293cece15","Type":"ContainerStarted","Data":"e2fb73a9f703a8624e50ba648bcc06c744030a1bbc967620f8e7fc96f689b2b6"}
Oct 30 01:49:14.324352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:14.324296    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" podStartSLOduration=2.324247303 podStartE2EDuration="2.324247303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:49:14.320230418 +0000 UTC m=+4062.349540648" watchObservedRunningTime="2026-10-30 01:49:14.324247303 +0000 UTC m=+4062.353557577"
Oct 30 01:49:15.057050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:15.057012    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:15.815181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:15.815146    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:17.614651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:17.614616    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:19.380250 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:19.380208    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:21.027791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:21.027748    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:22.747073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:22.747033    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:23.057274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:23.057182    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:23.075990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:23.075962    4577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:23.367343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:23.367248    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:49:24.513252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:24.513217    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:26.176858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:26.176816    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:26.661525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:26.661485    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:27.014705 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:49:27.014671    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:49:27.925713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:27.925673    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:29.737819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:29.737782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:31.510864 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:31.510812    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:33.307785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:33.307754    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:35.251998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:35.251960    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:37.119944 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:37.119908    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:38.928160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:38.928123    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:40.745465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:40.745419    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:42.556019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:42.555913    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:44.319587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:44.319532    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:46.162992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:46.162946    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:47.557985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:47.557947    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:47.675613 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:47.675574    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:47.903542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:47.903428    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:49.625397 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:49.625362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:51.316124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:51.316070    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:53.009231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:53.009196    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:54.853463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:54.853423    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:56.596792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:56.596751    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:49:57.014641 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:49:57.014598    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:49:58.404573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:49:58.404537    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:00.065895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:00.065861    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:01.818595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:01.818561    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:03.602439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:03.602400    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:05.406400 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:05.406368    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:07.241285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:07.241236    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:09.114696 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:09.114662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:11.021914 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:11.021884    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:12.817100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:12.817064    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:14.614270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:14.614235    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:16.431700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:16.431669    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:18.461996 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:18.461962    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:20.207459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:20.207424    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:21.990279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:21.990241    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:23.851979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:23.851942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:25.312773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:25.312737    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:26.011473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:26.011435    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:26.123809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:26.123776    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:26.660238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:26.660206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:26.782904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:26.782873    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:27.013983 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:50:27.013939    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:50:28.129648 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:28.129613    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:29.998030 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:29.997991    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:31.813365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:31.813316    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:33.836589 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:33.836558    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:34.582581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:34.582543    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:34.736201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:34.736140    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:34.925448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:34.925347    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:36.045802 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:36.045767    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:36.309752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:36.309654    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:38.005457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:38.005421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:40.028503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:40.028460    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:41.735714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:41.735662    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:43.704432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:43.704393    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:45.519559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:45.519520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:46.469666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:46.469617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:46.608990 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:46.608890    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:47.215807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:47.215743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:48.006876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:48.005411    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:49.824732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:49.824695    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:51.620223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:51.620191    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:53.471276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:53.471234    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:55.213769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:55.213739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:57.030442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:57.030362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:50:57.046965 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:50:57.041380    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:50:58.933538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:50:58.933431    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:00.992109 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:00.992079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:02.723205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:02.723174    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:02.963869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:02.959901    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:04.849219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:04.849177    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:06.622749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:06.622715    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:08.378920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:08.378881    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:10.186499 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:10.183944    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:12.005980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:12.005920    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:13.909794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:13.909721    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:14.302036 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:14.302002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:15.826411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:15.826376    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:17.571154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:17.571119    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:19.413315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:19.413280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:21.377554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:21.377519    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:23.369410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:23.369366    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:25.363553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:25.363495    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:26.935414 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:26.935383    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:27.015406 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:51:27.015371    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:51:27.286868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:27.286800    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:29.121240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:29.121211    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:31.105875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:31.099682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:33.074801 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:33.074760    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:35.041127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:35.041090    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:35.154991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:35.154955    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:35.669504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:35.669467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:37.033913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:37.033873    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:38.841875 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:38.838719    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:40.729045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:40.729010    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:42.630769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:42.630733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:43.431314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:43.431270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:43.555287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:43.555243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:43.759509 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:43.759471    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:44.604434 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:44.604392    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:46.270357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:46.270320    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:46.716758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:46.716669    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:48.520924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:48.520889    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:50.363895 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:50.359607    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:52.148726 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:52.148694    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:53.915118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:53.915074    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:55.003374 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:55.003291    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:55.139720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:55.139533    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:55.719289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:55.719245    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:57.015290 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:51:57.015253    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:51:57.458752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:57.458512    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:51:59.233013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:51:59.232984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:01.092496 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:01.092455    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:02.832603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:02.832571    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:04.652295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:04.652258    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:05.698122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:05.698071    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:06.582233 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:06.582171    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:08.643214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:08.643179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:10.505356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:10.505305    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:12.545758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:12.545710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:14.718237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:14.718208    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:16.461920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:16.461810    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:16.597560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:16.597488    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:18.223752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:18.223708    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:19.764757 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:19.764724    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:19.877718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:19.877688    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:21.609201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:21.609171    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:23.336627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:23.336585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:25.104486 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:25.104457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:26.579518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:26.579489    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:27.022726 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:52:27.022676    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:52:27.074525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:27.074488    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:27.199299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:27.199156    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:27.314348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:27.314267    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:27.719586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:27.719497    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:29.111873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:29.111800    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:30.843545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:30.843500    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:32.623992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:32.623830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:32.741629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:32.741597    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:34.438699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:34.438666    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:35.084526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:35.084491    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:35.194513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:35.194480    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:35.474709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:35.474615    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:35.968072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:35.968039    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"]
Oct 30 01:52:36.009031 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:36.008999    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:36.380811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:36.380764    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:38.167624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:38.167590    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:39.995342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:39.986212    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:41.664771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:41.664742    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:43.614360 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:43.614279    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:45.397902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:45.396247    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:46.764647 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:46.764612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:46.895916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:46.888612    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:47.323896 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:47.323857    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:47.764728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:47.764614    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:47.888580 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:47.888536    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:49.260334 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:49.260300    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:50.828314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:50.828273    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:50.991207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:50.991157    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" podUID="f8c1a9a9-9b24-446b-b543-99e293cece15" containerName="main" containerID="cri-o://d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c" gracePeriod=45
Oct 30 01:52:51.364097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.364048    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" podUID="f8c1a9a9-9b24-446b-b543-99e293cece15" containerName="main" probeResult="failure" output="Get \"https://10.134.1.20:8000/ready\": dial tcp 10.134.1.20:8000: connect: connection refused"
Oct 30 01:52:51.395933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.387599    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:51.516221 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.516187    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:51.663762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.663726    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:52:51.734817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.734775    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f\" (UniqueName: \"kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f\") pod \"f8c1a9a9-9b24-446b-b543-99e293cece15\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") "
Oct 30 01:52:51.735017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.734826    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs\") pod \"f8c1a9a9-9b24-446b-b543-99e293cece15\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") "
Oct 30 01:52:51.735017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.734920    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache\") pod \"f8c1a9a9-9b24-446b-b543-99e293cece15\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") "
Oct 30 01:52:51.735017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.734970    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir\") pod \"f8c1a9a9-9b24-446b-b543-99e293cece15\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") "
Oct 30 01:52:51.735208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735114    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home\") pod \"f8c1a9a9-9b24-446b-b543-99e293cece15\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") "
Oct 30 01:52:51.735208 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735198    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm\") pod \"f8c1a9a9-9b24-446b-b543-99e293cece15\" (UID: \"f8c1a9a9-9b24-446b-b543-99e293cece15\") "
Oct 30 01:52:51.735338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735199    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache" pod "f8c1a9a9-9b24-446b-b543-99e293cece15" (UID: "f8c1a9a9-9b24-446b-b543-99e293cece15"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:52:51.735338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735295    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir" pod "f8c1a9a9-9b24-446b-b543-99e293cece15" (UID: "f8c1a9a9-9b24-446b-b543-99e293cece15"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:52:51.735444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735406    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home" pod "f8c1a9a9-9b24-446b-b543-99e293cece15" (UID: "f8c1a9a9-9b24-446b-b543-99e293cece15"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:52:51.735885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735837    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:52:51.736086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735889    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:52:51.736086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.735906    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:52:51.737870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.737815    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs" pod "f8c1a9a9-9b24-446b-b543-99e293cece15" (UID: "f8c1a9a9-9b24-446b-b543-99e293cece15"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:52:51.738145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.738114    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm" pod "f8c1a9a9-9b24-446b-b543-99e293cece15" (UID: "f8c1a9a9-9b24-446b-b543-99e293cece15"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:52:51.738577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.738551    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f" pod "f8c1a9a9-9b24-446b-b543-99e293cece15" (UID: "f8c1a9a9-9b24-446b-b543-99e293cece15"). InnerVolumeSpecName "kube-api-access-cfc6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:52:51.837796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.837763    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8c1a9a9-9b24-446b-b543-99e293cece15-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:52:51.837796 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.837799    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfc6f\" (UniqueName: \"kubernetes.io/projected/f8c1a9a9-9b24-446b-b543-99e293cece15-kube-api-access-cfc6f\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:52:51.838329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.837814    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8c1a9a9-9b24-446b-b543-99e293cece15-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:52:51.863415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.863382    4577 generic.go:356] "Generic (PLEG): container finished" podID="f8c1a9a9-9b24-446b-b543-99e293cece15" containerID="d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c" exitCode=0
Oct 30 01:52:51.863415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.863422    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" event={"ID":"f8c1a9a9-9b24-446b-b543-99e293cece15","Type":"ContainerDied","Data":"d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c"}
Oct 30 01:52:51.863693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.863427    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"
Oct 30 01:52:51.863693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.863454    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw" event={"ID":"f8c1a9a9-9b24-446b-b543-99e293cece15","Type":"ContainerDied","Data":"e2fb73a9f703a8624e50ba648bcc06c744030a1bbc967620f8e7fc96f689b2b6"}
Oct 30 01:52:51.863693 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.863472    4577 scope.go:122] "RemoveContainer" containerID="d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c"
Oct 30 01:52:51.877982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.877956    4577 scope.go:122] "RemoveContainer" containerID="d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c"
Oct 30 01:52:51.878391 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:52:51.878365    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c\": container with ID starting with d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c not found: ID does not exist" containerID="d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c"
Oct 30 01:52:51.878446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.878406    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c"} err="failed to get container status \"d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c\": rpc error: code = NotFound desc = could not find container \"d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c\": container with ID starting with d64b31387e0f7959b04310ee259b5b60d03da2d611b45d23e88b9265f0b4f57c not found: ID does not exist"
Oct 30 01:52:51.922834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.922748    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"]
Oct 30 01:52:51.926680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:51.926655    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-9da49f/rate-test-model-9da49f-kserve-66cfcb4ffc-dgkmw"]
Oct 30 01:52:52.016634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:52.016603    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:52.114494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:52.114467    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8c1a9a9-9b24-446b-b543-99e293cece15" path="/var/lib/kubelet/pods/f8c1a9a9-9b24-446b-b543-99e293cece15/volumes"
Oct 30 01:52:53.315314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:53.315280    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:55.007019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:55.006986    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:56.671020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:56.670974    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:57.015751 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:52:57.015712    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:52:58.636091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:58.636052    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:59.304883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:59.304829    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:52:59.909982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:52:59.909941    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:00.089214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:00.089171    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:00.183465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:00.183371    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"]
Oct 30 01:53:00.184828 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:00.184580    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy" containerID="cri-o://61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607" gracePeriod=30
Oct 30 01:53:00.360868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:00.352425    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:00.922643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:00.922606    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:01.281017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:01.280976    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:02.896707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:02.896672    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:04.375093 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:04.375056    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"]
Oct 30 01:53:04.375660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:04.375268    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft" podUID="38283fe4-8bfc-442b-b730-87b614feed0c" containerName="maas-api" containerID="cri-o://9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97" gracePeriod=30
Oct 30 01:53:04.441987 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:04.441954    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:04.995933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:04.971075    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:53:05.032298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:04.980689    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:05.063700 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.063661    4577 generic.go:356] "Generic (PLEG): container finished" podID="38283fe4-8bfc-442b-b730-87b614feed0c" containerID="9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97" exitCode=0
Oct 30 01:53:05.065960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.064018    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft" event={"ID":"38283fe4-8bfc-442b-b730-87b614feed0c","Type":"ContainerDied","Data":"9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97"}
Oct 30 01:53:05.065960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.064084    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft" event={"ID":"38283fe4-8bfc-442b-b730-87b614feed0c","Type":"ContainerDied","Data":"5d5e311db776844daea82ef523b0a2fb99d0b154f7db0cd82f7719aebde31149"}
Oct 30 01:53:05.065960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.064118    4577 scope.go:122] "RemoveContainer" containerID="9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97"
Oct 30 01:53:05.065960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.064314    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"
Oct 30 01:53:05.117536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.099050    4577 scope.go:122] "RemoveContainer" containerID="9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97"
Oct 30 01:53:05.117536 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:05.104678    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97\": container with ID starting with 9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97 not found: ID does not exist" containerID="9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97"
Oct 30 01:53:05.117536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.104728    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97"} err="failed to get container status \"9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97\": rpc error: code = NotFound desc = could not find container \"9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97\": container with ID starting with 9cfea3b7e09ce7725eae4a8839932eff9b7fde60f53f19af49507363f008ef97 not found: ID does not exist"
Oct 30 01:53:05.145508 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.145468    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:05.230870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.212904    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx\" (UniqueName: \"kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx\") pod \"38283fe4-8bfc-442b-b730-87b614feed0c\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") "
Oct 30 01:53:05.230870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.213022    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs\") pod \"38283fe4-8bfc-442b-b730-87b614feed0c\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") "
Oct 30 01:53:05.230870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.213063    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca\") pod \"38283fe4-8bfc-442b-b730-87b614feed0c\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") "
Oct 30 01:53:05.230870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.213223    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls\") pod \"38283fe4-8bfc-442b-b730-87b614feed0c\" (UID: \"38283fe4-8bfc-442b-b730-87b614feed0c\") "
Oct 30 01:53:05.243128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.243087    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca" pod "38283fe4-8bfc-442b-b730-87b614feed0c" (UID: "38283fe4-8bfc-442b-b730-87b614feed0c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:05.261364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.261271    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs" pod "38283fe4-8bfc-442b-b730-87b614feed0c" (UID: "38283fe4-8bfc-442b-b730-87b614feed0c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:53:05.262124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.262087    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls" pod "38283fe4-8bfc-442b-b730-87b614feed0c" (UID: "38283fe4-8bfc-442b-b730-87b614feed0c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:53:05.263000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.262946    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx" pod "38283fe4-8bfc-442b-b730-87b614feed0c" (UID: "38283fe4-8bfc-442b-b730-87b614feed0c"). InnerVolumeSpecName "kube-api-access-fhpkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:05.315291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.315258    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhpkx\" (UniqueName: \"kubernetes.io/projected/38283fe4-8bfc-442b-b730-87b614feed0c-kube-api-access-fhpkx\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.315291 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.315292    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.315462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.315306    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38283fe4-8bfc-442b-b730-87b614feed0c-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.315462 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.315318    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38283fe4-8bfc-442b-b730-87b614feed0c-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.549114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.549010    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"]
Oct 30 01:53:05.575634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.575540    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-df4db1-6d68bf75fb-lc2ft"]
Oct 30 01:53:05.589354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.589302    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:05.693276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.693249    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:53:05.726779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.726746    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.726807    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.726839    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.726931    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.726955    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727026    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727075    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727126    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:05.727262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727234    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727314    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert\") pod \"5087d495-29d9-43a8-872c-10a43a92f05d\" (UID: \"5087d495-29d9-43a8-872c-10a43a92f05d\") "
Oct 30 01:53:05.727495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727317    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:05.727495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727380    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:05.727663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727493    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:05.727767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.727744    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:05.728256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.728230    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5087d495-29d9-43a8-872c-10a43a92f05d-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.728351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.728261    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.728351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.728276    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.728351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.728290    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.728351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.728305    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.729513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.729482    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:05.729631 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.729575    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:53:05.729681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.729645    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "kube-api-access-jb4dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:05.729681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.729667    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token" pod "5087d495-29d9-43a8-872c-10a43a92f05d" (UID: "5087d495-29d9-43a8-872c-10a43a92f05d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:05.829780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.829685    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5087d495-29d9-43a8-872c-10a43a92f05d-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.829780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.829717    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.829780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.829726    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb4dl\" (UniqueName: \"kubernetes.io/projected/5087d495-29d9-43a8-872c-10a43a92f05d-kube-api-access-jb4dl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:05.829780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:05.829736    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5087d495-29d9-43a8-872c-10a43a92f05d-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:06.072773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.072735    4577 generic.go:356] "Generic (PLEG): container finished" podID="5087d495-29d9-43a8-872c-10a43a92f05d" containerID="61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607" exitCode=0
Oct 30 01:53:06.073009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.072804    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"
Oct 30 01:53:06.073009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.072814    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" event={"ID":"5087d495-29d9-43a8-872c-10a43a92f05d","Type":"ContainerDied","Data":"61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607"}
Oct 30 01:53:06.073009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.072936    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g" event={"ID":"5087d495-29d9-43a8-872c-10a43a92f05d","Type":"ContainerDied","Data":"983952a04d1d0e4346f1ea6078880854d39d37fcf0516f0768718d8d3a303866"}
Oct 30 01:53:06.073009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.072965    4577 scope.go:122] "RemoveContainer" containerID="61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607"
Oct 30 01:53:06.085433 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.085413    4577 scope.go:122] "RemoveContainer" containerID="2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"
Oct 30 01:53:06.096323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.096305    4577 scope.go:122] "RemoveContainer" containerID="61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607"
Oct 30 01:53:06.096628 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:06.096605    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607\": container with ID starting with 61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607 not found: ID does not exist" containerID="61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607"
Oct 30 01:53:06.096743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.096634    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607"} err="failed to get container status \"61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607\": rpc error: code = NotFound desc = could not find container \"61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607\": container with ID starting with 61df6304501ddc986e3ab871e3ec982fd26c4aa0689d149a0571cc3e9d8ba607 not found: ID does not exist"
Oct 30 01:53:06.096743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.096653    4577 scope.go:122] "RemoveContainer" containerID="2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"
Oct 30 01:53:06.096939 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:06.096919    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d\": container with ID starting with 2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d not found: ID does not exist" containerID="2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"
Oct 30 01:53:06.097001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.096944    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d"} err="failed to get container status \"2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d\": rpc error: code = NotFound desc = could not find container \"2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d\": container with ID starting with 2d99fe18a095bb0aef78c6a5b080d10747967b32776b8bf906889bb227d9307d not found: ID does not exist"
Oct 30 01:53:06.113474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.113449    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38283fe4-8bfc-442b-b730-87b614feed0c" path="/var/lib/kubelet/pods/38283fe4-8bfc-442b-b730-87b614feed0c/volumes"
Oct 30 01:53:06.133384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.133349    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"]
Oct 30 01:53:06.137432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:06.137407    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-499482-openshift-default-6fff6f7b5-7t85g"]
Oct 30 01:53:07.015742 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:07.015710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:08.113856 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:08.113799    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" path="/var/lib/kubelet/pods/5087d495-29d9-43a8-872c-10a43a92f05d/volumes"
Oct 30 01:53:08.825386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:08.825331    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:10.652872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:10.647967    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:11.570870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:11.566375    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:11.677887 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:11.677836    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:12.148699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:12.148604    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:12.283329 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:12.283270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:12.461361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:12.461240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:13.241874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:13.241816    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:13.360299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:13.360267    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:13.596780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:13.596692    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"]
Oct 30 01:53:13.596986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:13.596940    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy" containerID="cri-o://def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777" gracePeriod=30
Oct 30 01:53:13.598015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:13.597987    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:14.267560 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:14.267522    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:16.241176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:16.241143    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:17.884069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:17.884022    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:18.443086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.435793    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:18.604206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.604167    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:18.919484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.919460    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:53:18.921762 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.921736    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.921924 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.921812    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.921977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.921943    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.921977 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.921969    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.922070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922047    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:18.922133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922086    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:18.922195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922122    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.922253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922216    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.922305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922251    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:18.922305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922262    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.922424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922313    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:18.922424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922342    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:18.922590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922569    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:18.922994 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922979    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b1183c-5416-461c-9375-7e377bb762a1-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:18.923059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.922996    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:18.923059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.923005    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:18.923059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.923014    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:18.923059 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.923021    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:18.924536 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.924516    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "kube-api-access-rtbkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:18.924770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.924751    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:53:18.924897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:18.924769    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:19.024554 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.024469    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token\") pod \"c4b1183c-5416-461c-9375-7e377bb762a1\" (UID: \"c4b1183c-5416-461c-9375-7e377bb762a1\") "
Oct 30 01:53:19.025096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.025074    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtbkj\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-kube-api-access-rtbkj\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:19.025096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.025093    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b1183c-5416-461c-9375-7e377bb762a1-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:19.025304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.025103    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b1183c-5416-461c-9375-7e377bb762a1-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:19.026832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.026806    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token" pod "c4b1183c-5416-461c-9375-7e377bb762a1" (UID: "c4b1183c-5416-461c-9375-7e377bb762a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:19.082057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.082022    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:19.127038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.127005    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b1183c-5416-461c-9375-7e377bb762a1-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:19.271784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.271751    4577 generic.go:356] "Generic (PLEG): container finished" podID="c4b1183c-5416-461c-9375-7e377bb762a1" containerID="def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777" exitCode=0
Oct 30 01:53:19.272014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.271820    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"
Oct 30 01:53:19.272014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.271832    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" event={"ID":"c4b1183c-5416-461c-9375-7e377bb762a1","Type":"ContainerDied","Data":"def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777"}
Oct 30 01:53:19.272014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.271904    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r" event={"ID":"c4b1183c-5416-461c-9375-7e377bb762a1","Type":"ContainerDied","Data":"e4380bcdf261505a97ee1a206a60aea81c960248c4cd36e3942e7086a5328539"}
Oct 30 01:53:19.272014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.271927    4577 scope.go:122] "RemoveContainer" containerID="def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777"
Oct 30 01:53:19.285045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.285024    4577 scope.go:122] "RemoveContainer" containerID="5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"
Oct 30 01:53:19.296411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.296395    4577 scope.go:122] "RemoveContainer" containerID="def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777"
Oct 30 01:53:19.296687 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:19.296669    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777\": container with ID starting with def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777 not found: ID does not exist" containerID="def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777"
Oct 30 01:53:19.296769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.296694    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777"} err="failed to get container status \"def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777\": rpc error: code = NotFound desc = could not find container \"def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777\": container with ID starting with def939ce5be769d67b88490f1c615074aad650ebb83c656a73854e1e8194b777 not found: ID does not exist"
Oct 30 01:53:19.296769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.296711    4577 scope.go:122] "RemoveContainer" containerID="5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"
Oct 30 01:53:19.296986 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:19.296971    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c\": container with ID starting with 5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c not found: ID does not exist" containerID="5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"
Oct 30 01:53:19.297032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.296991    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c"} err="failed to get container status \"5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c\": rpc error: code = NotFound desc = could not find container \"5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c\": container with ID starting with 5474d19d814fab5f0c8718eb86f4aa6d8b6bc5c765cac6a826daa7634837430c not found: ID does not exist"
Oct 30 01:53:19.331411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.331382    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"]
Oct 30 01:53:19.345201 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:19.345173    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-df4db1-openshift-default-795cbf9c5f-tdj6r"]
Oct 30 01:53:20.112540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:20.112509    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" path="/var/lib/kubelet/pods/c4b1183c-5416-461c-9375-7e377bb762a1/volumes"
Oct 30 01:53:20.444705 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:20.444627    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:22.179300 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:22.179263    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:23.907552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:23.907513    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:24.846957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:24.846922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:25.010972 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:25.010942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:25.868860 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:25.868814    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:26.525584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:26.525545    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:26.652148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:26.652116    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:26.876472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:26.876378    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"]
Oct 30 01:53:26.877679 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:26.877496    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" containerID="cri-o://c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3" gracePeriod=30
Oct 30 01:53:26.904070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:26.904039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:27.204908 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:27.204796    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:53:27.209638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:27.209607    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:27.824616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:27.824582    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:29.676988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:29.676942    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:31.513445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:31.513405    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:31.599382 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:31.599324    4577 patch_prober.go:29] interesting pod/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:53:31.599610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:31.599427    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:53:32.171303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.171277    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:53:32.298540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298503    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.298770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298586    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.298770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298611    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.298770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298648    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.298770 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298696    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.299035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298781    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.299035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298891    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.299035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.298959    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.299035 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.299009    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token\") pod \"da1bb547-5123-41b1-8a31-86b0b33b4837\" (UID: \"da1bb547-5123-41b1-8a31-86b0b33b4837\") "
Oct 30 01:53:32.299587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.299557    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:32.299945 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.299917    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:32.300235 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.300199    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:32.300354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.300268    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:32.300550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.300423    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:32.303274 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.303248    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:32.303385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.303329    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:53:32.303982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.303956    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:32.308228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.308204    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c" pod "da1bb547-5123-41b1-8a31-86b0b33b4837" (UID: "da1bb547-5123-41b1-8a31-86b0b33b4837"). InnerVolumeSpecName "kube-api-access-vdj6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401358    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401385    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401396    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401404    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401411    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401419    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1bb547-5123-41b1-8a31-86b0b33b4837-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401426    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401433    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1bb547-5123-41b1-8a31-86b0b33b4837-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.401443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.401441    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdj6c\" (UniqueName: \"kubernetes.io/projected/da1bb547-5123-41b1-8a31-86b0b33b4837-kube-api-access-vdj6c\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:32.509450 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.509418    4577 generic.go:356] "Generic (PLEG): container finished" podID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerID="c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3" exitCode=0
Oct 30 01:53:32.509667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.509470    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" event={"ID":"da1bb547-5123-41b1-8a31-86b0b33b4837","Type":"ContainerDied","Data":"c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3"}
Oct 30 01:53:32.509667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.509517    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb" event={"ID":"da1bb547-5123-41b1-8a31-86b0b33b4837","Type":"ContainerDied","Data":"b9fe4069c4013b59db05929b1b4b9f1d3fafdafdbe80f02f89f84bef8adc1a1d"}
Oct 30 01:53:32.509667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.509538    4577 scope.go:122] "RemoveContainer" containerID="c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3"
Oct 30 01:53:32.509667 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.509471    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"
Oct 30 01:53:32.522044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.522014    4577 scope.go:122] "RemoveContainer" containerID="e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"
Oct 30 01:53:32.534637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.534607    4577 scope.go:122] "RemoveContainer" containerID="c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3"
Oct 30 01:53:32.534961 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:32.534943    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3\": container with ID starting with c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3 not found: ID does not exist" containerID="c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3"
Oct 30 01:53:32.535046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.534969    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3"} err="failed to get container status \"c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3\": rpc error: code = NotFound desc = could not find container \"c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3\": container with ID starting with c04c6081f7c0d04b04d041a0f66b3994e06c2ff569a548cfc61f379128db9fc3 not found: ID does not exist"
Oct 30 01:53:32.535046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.534998    4577 scope.go:122] "RemoveContainer" containerID="e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"
Oct 30 01:53:32.535249 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:32.535233    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9\": container with ID starting with e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9 not found: ID does not exist" containerID="e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"
Oct 30 01:53:32.535294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.535253    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9"} err="failed to get container status \"e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9\": rpc error: code = NotFound desc = could not find container \"e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9\": container with ID starting with e2904390ca72a4a7004bee1eaad801f99e1fa0044fc0dc9fd1af8f4c8d26d6d9 not found: ID does not exist"
Oct 30 01:53:32.568946 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.568912    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"]
Oct 30 01:53:32.582192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:32.582151    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-1cb829-openshift-default-7959b897cc-wxdqb"]
Oct 30 01:53:33.354168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:33.354123    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:33.482021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:33.481983    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:33.844678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:33.844645    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:34.112391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:34.112313    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" path="/var/lib/kubelet/pods/da1bb547-5123-41b1-8a31-86b0b33b4837/volumes"
Oct 30 01:53:35.089699 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:35.089664    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:36.836538 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:36.836499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:38.160970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.160935    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:38.287774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.287709    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:38.336013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.335964    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"]
Oct 30 01:53:38.336255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.336229    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api" containerID="cri-o://41caef17a0c3afe2e551b976f69a1c40f0f47268ac280c58b37cb9184b2ad617" gracePeriod=30
Oct 30 01:53:38.477339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.477292    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:38.575207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.574967    4577 generic.go:356] "Generic (PLEG): container finished" podID="62f086f0-79fa-412f-a322-3426ba607137" containerID="41caef17a0c3afe2e551b976f69a1c40f0f47268ac280c58b37cb9184b2ad617" exitCode=0
Oct 30 01:53:38.575207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.575030    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" event={"ID":"62f086f0-79fa-412f-a322-3426ba607137","Type":"ContainerDied","Data":"41caef17a0c3afe2e551b976f69a1c40f0f47268ac280c58b37cb9184b2ad617"}
Oct 30 01:53:38.575207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.575085    4577 scope.go:122] "RemoveContainer" containerID="47fe6af196cb4f16690e259b690d7286edda4a2c070a8c028f3cc6dd43d82964"
Oct 30 01:53:38.700920 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.700889    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:53:38.848868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.840189    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz\" (UniqueName: \"kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz\") pod \"62f086f0-79fa-412f-a322-3426ba607137\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") "
Oct 30 01:53:38.848868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.840281    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs\") pod \"62f086f0-79fa-412f-a322-3426ba607137\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") "
Oct 30 01:53:38.848868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.840361    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls\") pod \"62f086f0-79fa-412f-a322-3426ba607137\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") "
Oct 30 01:53:38.848868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.840448    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca\") pod \"62f086f0-79fa-412f-a322-3426ba607137\" (UID: \"62f086f0-79fa-412f-a322-3426ba607137\") "
Oct 30 01:53:38.848868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.841782    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca" pod "62f086f0-79fa-412f-a322-3426ba607137" (UID: "62f086f0-79fa-412f-a322-3426ba607137"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:38.974873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:38.947503    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62f086f0-79fa-412f-a322-3426ba607137-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:39.012006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.011950    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz" pod "62f086f0-79fa-412f-a322-3426ba607137" (UID: "62f086f0-79fa-412f-a322-3426ba607137"). InnerVolumeSpecName "kube-api-access-rphzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:39.019632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.019546    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:39.089130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.087186    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rphzz\" (UniqueName: \"kubernetes.io/projected/62f086f0-79fa-412f-a322-3426ba607137-kube-api-access-rphzz\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:39.124930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.116993    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs" pod "62f086f0-79fa-412f-a322-3426ba607137" (UID: "62f086f0-79fa-412f-a322-3426ba607137"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:53:39.127261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.127221    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls" pod "62f086f0-79fa-412f-a322-3426ba607137" (UID: "62f086f0-79fa-412f-a322-3426ba607137"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:53:39.159373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.159318    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:39.188491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.188449    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:39.188491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.188477    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62f086f0-79fa-412f-a322-3426ba607137-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:39.585087 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.585058    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"
Oct 30 01:53:39.585323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.585093    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx" event={"ID":"62f086f0-79fa-412f-a322-3426ba607137","Type":"ContainerDied","Data":"61c370220e2f5c131a7620fa266516ee9d353c791342c54a8d34225cefe08667"}
Oct 30 01:53:39.585323 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.585163    4577 scope.go:122] "RemoveContainer" containerID="41caef17a0c3afe2e551b976f69a1c40f0f47268ac280c58b37cb9184b2ad617"
Oct 30 01:53:39.648206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.648175    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"]
Oct 30 01:53:39.661674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.661646    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-dd80cb-65889fd454-hxhtx"]
Oct 30 01:53:39.676245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:39.676210    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:40.112819 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:40.112781    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62f086f0-79fa-412f-a322-3426ba607137" path="/var/lib/kubelet/pods/62f086f0-79fa-412f-a322-3426ba607137/volumes"
Oct 30 01:53:40.674461 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:40.674426    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:41.264271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:41.264238    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:41.382651 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:41.382613    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:41.814475 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:41.814412    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:43.199098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:43.199059    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:44.762818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:44.762782    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:44.978310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:44.978275    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:46.405732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:46.405694    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:46.521723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:46.521685    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:46.744219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:46.744183    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"]
Oct 30 01:53:46.744490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:46.744452    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy" containerID="cri-o://74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4" gracePeriod=30
Oct 30 01:53:46.805469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:46.805425    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:47.080407 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:47.080314    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:47.251688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:47.251644    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:47.751172 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:47.751139    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:49.059787 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:49.059754    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:49.176270 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:49.176240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:49.294100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:49.294063    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:49.364727 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:49.364641    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"]
Oct 30 01:53:49.364967 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:49.364937    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy" containerID="cri-o://2c4a5ebea899c1fc6c219385d1f62560fe692da78e809b5f71e0828ce15d3b88" gracePeriod=30
Oct 30 01:53:49.418032 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:49.417999    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:51.053255 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:51.053222    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:52.021064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.020999    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:53:52.066309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066272    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066313    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066376    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066434    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066559    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066582    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066645    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066674    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066689    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066731    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy\") pod \"61823c04-7687-49d3-95ef-915421584fa0\" (UID: \"61823c04-7687-49d3-95ef-915421584fa0\") "
Oct 30 01:53:52.066839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.066780    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:52.067385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067027    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:52.067385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067047    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:52.067527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067501    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:52.067740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067722    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.067784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067748    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.067784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067763    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.067784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067776    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61823c04-7687-49d3-95ef-915421584fa0-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.067910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.067789    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.069441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.069416    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:53:52.069557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.069472    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "kube-api-access-m79r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:52.070050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.070024    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:52.070214 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.070201    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy" pod "61823c04-7687-49d3-95ef-915421584fa0" (UID: "61823c04-7687-49d3-95ef-915421584fa0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:52.168932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.168896    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.168932 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.168932    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61823c04-7687-49d3-95ef-915421584fa0-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.169191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.168948    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m79r2\" (UniqueName: \"kubernetes.io/projected/61823c04-7687-49d3-95ef-915421584fa0-kube-api-access-m79r2\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.169191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.168964    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61823c04-7687-49d3-95ef-915421584fa0-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:52.688130 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.688096    4577 generic.go:356] "Generic (PLEG): container finished" podID="61823c04-7687-49d3-95ef-915421584fa0" containerID="74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4" exitCode=0
Oct 30 01:53:52.688298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.688157    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"
Oct 30 01:53:52.688298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.688180    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" event={"ID":"61823c04-7687-49d3-95ef-915421584fa0","Type":"ContainerDied","Data":"74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4"}
Oct 30 01:53:52.688298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.688241    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2" event={"ID":"61823c04-7687-49d3-95ef-915421584fa0","Type":"ContainerDied","Data":"d342f6cd073f80c4d6e77a18e24741ee04ce0480eebc764d2c8eb4ae4d0b460e"}
Oct 30 01:53:52.688298 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.688265    4577 scope.go:122] "RemoveContainer" containerID="74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4"
Oct 30 01:53:52.699098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.699078    4577 scope.go:122] "RemoveContainer" containerID="158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"
Oct 30 01:53:52.708952 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.708934    4577 scope.go:122] "RemoveContainer" containerID="74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4"
Oct 30 01:53:52.709229 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:52.709207    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4\": container with ID starting with 74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4 not found: ID does not exist" containerID="74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4"
Oct 30 01:53:52.709303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.709244    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4"} err="failed to get container status \"74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4\": rpc error: code = NotFound desc = could not find container \"74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4\": container with ID starting with 74414456071a7347145e3103ec26c18cf37e42f4d0ccc5f0e2262ceca2edc8f4 not found: ID does not exist"
Oct 30 01:53:52.709303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.709269    4577 scope.go:122] "RemoveContainer" containerID="158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"
Oct 30 01:53:52.709582 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:52.709564    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d\": container with ID starting with 158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d not found: ID does not exist" containerID="158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"
Oct 30 01:53:52.709625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.709589    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d"} err="failed to get container status \"158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d\": rpc error: code = NotFound desc = could not find container \"158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d\": container with ID starting with 158c38921c5b975a24715b653e8d0c22d3b91dc85367ea66ec58e773ea3b009d not found: ID does not exist"
Oct 30 01:53:52.784872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.782031    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:52.793411 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.793380    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"]
Oct 30 01:53:52.799305 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:52.799280    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-dd80cb-openshift-default-9d596b745-259c2"]
Oct 30 01:53:54.111886 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.111829    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61823c04-7687-49d3-95ef-915421584fa0" path="/var/lib/kubelet/pods/61823c04-7687-49d3-95ef-915421584fa0/volumes"
Oct 30 01:53:54.504311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.504270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:54.635292 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.634451    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:54.708485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.708444    4577 generic.go:356] "Generic (PLEG): container finished" podID="aa6ec37d-8703-4491-a295-91eb0e031794" containerID="2c4a5ebea899c1fc6c219385d1f62560fe692da78e809b5f71e0828ce15d3b88" exitCode=0
Oct 30 01:53:54.708636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.708518    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" event={"ID":"aa6ec37d-8703-4491-a295-91eb0e031794","Type":"ContainerDied","Data":"2c4a5ebea899c1fc6c219385d1f62560fe692da78e809b5f71e0828ce15d3b88"}
Oct 30 01:53:54.708636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.708576    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp" event={"ID":"aa6ec37d-8703-4491-a295-91eb0e031794","Type":"ContainerDied","Data":"4fd72329dbfc78629b7323c0ff1f660b7a1d90935689eea87cc51efcf2eab75c"}
Oct 30 01:53:54.708636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.708592    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fd72329dbfc78629b7323c0ff1f660b7a1d90935689eea87cc51efcf2eab75c"
Oct 30 01:53:54.708636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.708605    4577 scope.go:122] "RemoveContainer" containerID="e0780bb0f3158782707471cc027dc31d76daf852dcf4aa79ec15fbce11d7ce2c"
Oct 30 01:53:54.720826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.720806    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.808585    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.808651    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.808727    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.808876    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.808942    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809018    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809052    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809096    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809141    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809219    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809270    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809409    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6\") pod \"aa6ec37d-8703-4491-a295-91eb0e031794\" (UID: \"aa6ec37d-8703-4491-a295-91eb0e031794\") "
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809534    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.809581    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810129    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810150    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810178    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810191    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810204    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6ec37d-8703-4491-a295-91eb0e031794-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810488    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"]
Oct 30 01:53:54.811664 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.810716    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy" containerID="cri-o://c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd" gracePeriod=30
Oct 30 01:53:54.818144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.817405    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:53:54.818144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.817433    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:54.818350 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.818247    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "kube-api-access-99ds6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:54.818755 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.818728    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy" pod "aa6ec37d-8703-4491-a295-91eb0e031794" (UID: "aa6ec37d-8703-4491-a295-91eb0e031794"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:53:54.831001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.830889    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:54.919146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.918051    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99ds6\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-kube-api-access-99ds6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.919146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.918097    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6ec37d-8703-4491-a295-91eb0e031794-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.919146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.918113    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6ec37d-8703-4491-a295-91eb0e031794-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:54.919146 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:54.918125    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6ec37d-8703-4491-a295-91eb0e031794-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:55.718145 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:55.718113    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"
Oct 30 01:53:55.838234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:55.838198    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"]
Oct 30 01:53:55.847068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:55.847039    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-c48628-openshift-default-784bcb8789-b6frp"]
Oct 30 01:53:56.112982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.112951    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" path="/var/lib/kubelet/pods/aa6ec37d-8703-4491-a295-91eb0e031794/volumes"
Oct 30 01:53:56.142581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.142544    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"]
Oct 30 01:53:56.142805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.142781    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" podUID="5aa19f12-8a54-4aad-9e86-b3344edac6db" containerName="maas-api" containerID="cri-o://7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6" gracePeriod=30
Oct 30 01:53:56.212825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.212786    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:56.512194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.512161    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:53:56.555986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.555935    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs\") pod \"5aa19f12-8a54-4aad-9e86-b3344edac6db\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") "
Oct 30 01:53:56.556199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.556055    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl\" (UniqueName: \"kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl\") pod \"5aa19f12-8a54-4aad-9e86-b3344edac6db\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") "
Oct 30 01:53:56.556285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.556264    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\") pod \"5aa19f12-8a54-4aad-9e86-b3344edac6db\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") "
Oct 30 01:53:56.556346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.556337    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca\") pod \"5aa19f12-8a54-4aad-9e86-b3344edac6db\" (UID: \"5aa19f12-8a54-4aad-9e86-b3344edac6db\") "
Oct 30 01:53:56.557380 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.557348    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca" pod "5aa19f12-8a54-4aad-9e86-b3344edac6db" (UID: "5aa19f12-8a54-4aad-9e86-b3344edac6db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:53:56.560136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.559786    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls" pod "5aa19f12-8a54-4aad-9e86-b3344edac6db" (UID: "5aa19f12-8a54-4aad-9e86-b3344edac6db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:53:56.561289 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.561224    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl" pod "5aa19f12-8a54-4aad-9e86-b3344edac6db" (UID: "5aa19f12-8a54-4aad-9e86-b3344edac6db"). InnerVolumeSpecName "kube-api-access-z5gkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:53:56.561534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.561501    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs" pod "5aa19f12-8a54-4aad-9e86-b3344edac6db" (UID: "5aa19f12-8a54-4aad-9e86-b3344edac6db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:53:56.687922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.687014    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:56.687922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.687058    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa19f12-8a54-4aad-9e86-b3344edac6db-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:56.687922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.687075    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa19f12-8a54-4aad-9e86-b3344edac6db-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:56.687922 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.687088    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5gkl\" (UniqueName: \"kubernetes.io/projected/5aa19f12-8a54-4aad-9e86-b3344edac6db-kube-api-access-z5gkl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:53:56.750892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.744234    4577 generic.go:356] "Generic (PLEG): container finished" podID="5aa19f12-8a54-4aad-9e86-b3344edac6db" containerID="7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6" exitCode=0
Oct 30 01:53:56.750892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.744294    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" event={"ID":"5aa19f12-8a54-4aad-9e86-b3344edac6db","Type":"ContainerDied","Data":"7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6"}
Oct 30 01:53:56.750892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.744344    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p" event={"ID":"5aa19f12-8a54-4aad-9e86-b3344edac6db","Type":"ContainerDied","Data":"de61739131ce47fc3600a979da3be12a35d26c1f9b222829ba3dce6de34a4cee"}
Oct 30 01:53:56.750892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.744374    4577 scope.go:122] "RemoveContainer" containerID="7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6"
Oct 30 01:53:56.750892 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.744554    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"
Oct 30 01:53:56.769037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.769011    4577 scope.go:122] "RemoveContainer" containerID="7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6"
Oct 30 01:53:56.769383 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:56.769358    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6\": container with ID starting with 7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6 not found: ID does not exist" containerID="7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6"
Oct 30 01:53:56.769496 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.769395    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6"} err="failed to get container status \"7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6\": rpc error: code = NotFound desc = could not find container \"7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6\": container with ID starting with 7c0555a2425d07bb8d0bf06107f69a56f49da3319723125f1d81832cec967bf6 not found: ID does not exist"
Oct 30 01:53:56.806442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.806415    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"]
Oct 30 01:53:56.808423 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.808384    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:56.811078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.811055    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-bdf6ca-68674b6cbd-cnw7p"]
Oct 30 01:53:56.948432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:56.948315    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:57.038185 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:53:57.038137    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:53:57.066381 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:57.066344    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:57.467722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:57.467670    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:57.977916 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:57.977879    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:58.096118 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:58.096088    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:53:58.114379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:58.114351    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5aa19f12-8a54-4aad-9e86-b3344edac6db" path="/var/lib/kubelet/pods/5aa19f12-8a54-4aad-9e86-b3344edac6db/volumes"
Oct 30 01:53:58.816545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:53:58.816512    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:00.120226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.120201    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:54:00.254040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254001    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254038    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254127    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254166    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254286    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254316    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254334    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:00.254483 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254406    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254403    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:00.254642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254485    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254561    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token\") pod \"f506ab8c-31f5-449c-9662-90d953bcb38c\" (UID: \"f506ab8c-31f5-449c-9662-90d953bcb38c\") "
Oct 30 01:54:00.254642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.254607    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:00.255083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.255026    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:00.255590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.255566    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.255676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.255627    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.255676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.255643    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f506ab8c-31f5-449c-9662-90d953bcb38c-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.255676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.255639    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:00.255676 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.255657    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.257199 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.257001    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:54:00.257324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.257219    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:00.257398 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.257341    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "kube-api-access-lwb8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:00.258070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.258051    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token" pod "f506ab8c-31f5-449c-9662-90d953bcb38c" (UID: "f506ab8c-31f5-449c-9662-90d953bcb38c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:00.314804 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.314770    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"]
Oct 30 01:54:00.357567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.357477    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.357567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.357509    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.357567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.357519    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.357567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.357526    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f506ab8c-31f5-449c-9662-90d953bcb38c-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.357567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.357535    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwb8x\" (UniqueName: \"kubernetes.io/projected/f506ab8c-31f5-449c-9662-90d953bcb38c-kube-api-access-lwb8x\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:00.498732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.498698    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:00.619445 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.619355    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:00.786792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.786751    4577 generic.go:356] "Generic (PLEG): container finished" podID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerID="c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd" exitCode=0
Oct 30 01:54:00.787003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.786813    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"
Oct 30 01:54:00.787003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.786813    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" event={"ID":"f506ab8c-31f5-449c-9662-90d953bcb38c","Type":"ContainerDied","Data":"c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd"}
Oct 30 01:54:00.787003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.786915    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6" event={"ID":"f506ab8c-31f5-449c-9662-90d953bcb38c","Type":"ContainerDied","Data":"b951b879d7028f220d177a0bdb478a36069892fd9f2443bb16b4980bac21e79f"}
Oct 30 01:54:00.787003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.786945    4577 scope.go:122] "RemoveContainer" containerID="c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd"
Oct 30 01:54:00.798424 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.798406    4577 scope.go:122] "RemoveContainer" containerID="06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"
Oct 30 01:54:00.808339 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.808320    4577 scope.go:122] "RemoveContainer" containerID="c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd"
Oct 30 01:54:00.808637 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:00.808619    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd\": container with ID starting with c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd not found: ID does not exist" containerID="c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd"
Oct 30 01:54:00.808712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.808648    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd"} err="failed to get container status \"c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd\": rpc error: code = NotFound desc = could not find container \"c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd\": container with ID starting with c8263be894054124b085c628196b8b42daf26a4f4eec0552b22d8d8d4a7e24fd not found: ID does not exist"
Oct 30 01:54:00.808712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.808665    4577 scope.go:122] "RemoveContainer" containerID="06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"
Oct 30 01:54:00.808918 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:00.808900    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6\": container with ID starting with 06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6 not found: ID does not exist" containerID="06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"
Oct 30 01:54:00.808964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.808923    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6"} err="failed to get container status \"06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6\": rpc error: code = NotFound desc = could not find container \"06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6\": container with ID starting with 06dd3c747073e0fc60519931a13989c6a0f36ffc85ffd7255c7ebdacca1309b6 not found: ID does not exist"
Oct 30 01:54:00.850739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.850710    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"]
Oct 30 01:54:00.856190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:00.856160    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-a47b9a-openshift-default-54dffbcfd9-7f5t6"]
Oct 30 01:54:02.111746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:02.111710    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" path="/var/lib/kubelet/pods/f506ab8c-31f5-449c-9662-90d953bcb38c/volumes"
Oct 30 01:54:02.291619 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:02.291585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:02.419568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:02.419481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:04.399986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:04.399948    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:05.315837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:05.315765    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerName="payload-processing" containerID="cri-o://c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28" gracePeriod=25
Oct 30 01:54:05.935375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:05.935340    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:06.050365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.050326    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:06.212785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.212689    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:06.402151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.402113    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"]
Oct 30 01:54:06.402406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.402375    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy" containerID="cri-o://4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3" gracePeriod=30
Oct 30 01:54:06.441639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.441608    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:06.866396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.866363    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:06.873390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:06.873357    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:54:08.040354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:08.040325    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:09.903817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:09.903783    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:11.687356 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.687288    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:54:11.770573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.770539    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771088    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771144    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771228    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771264    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771334    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771420    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771510    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.771730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.771556    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo\") pod \"92ddec96-952c-4a03-ae26-09aa2368fdfb\" (UID: \"92ddec96-952c-4a03-ae26-09aa2368fdfb\") "
Oct 30 01:54:11.773900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.773867    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:11.774194 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.774168    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:11.775571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.774379    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:11.775571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.774564    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:11.775571 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.774597    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:11.778561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.776034    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:11.778561 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.777730    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:54:11.781025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.780999    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "kube-api-access-82zd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:11.781202 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.781070    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token" pod "92ddec96-952c-4a03-ae26-09aa2368fdfb" (UID: "92ddec96-952c-4a03-ae26-09aa2368fdfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:11.830618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.830580    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:11.834673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.834647    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"]
Oct 30 01:54:11.874524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874492    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82zd6\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-kube-api-access-82zd6\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874523    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92ddec96-952c-4a03-ae26-09aa2368fdfb-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874539    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874550    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874561    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874573    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874583    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874593    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.874743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.874605    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92ddec96-952c-4a03-ae26-09aa2368fdfb-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:11.903207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.903171    4577 generic.go:356] "Generic (PLEG): container finished" podID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerID="4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3" exitCode=0
Oct 30 01:54:11.903379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.903217    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"
Oct 30 01:54:11.903379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.903255    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" event={"ID":"92ddec96-952c-4a03-ae26-09aa2368fdfb","Type":"ContainerDied","Data":"4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3"}
Oct 30 01:54:11.903379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.903343    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq" event={"ID":"92ddec96-952c-4a03-ae26-09aa2368fdfb","Type":"ContainerDied","Data":"e4b368103419d10db1462511b4b02f33d66792bca849cded9f3e66e865cbdca9"}
Oct 30 01:54:11.903379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.903371    4577 scope.go:122] "RemoveContainer" containerID="4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3"
Oct 30 01:54:11.914524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.914507    4577 scope.go:122] "RemoveContainer" containerID="52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"
Oct 30 01:54:11.924223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.924205    4577 scope.go:122] "RemoveContainer" containerID="4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3"
Oct 30 01:54:11.924483 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:11.924463    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3\": container with ID starting with 4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3 not found: ID does not exist" containerID="4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3"
Oct 30 01:54:11.924532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.924492    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3"} err="failed to get container status \"4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3\": rpc error: code = NotFound desc = could not find container \"4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3\": container with ID starting with 4e18f19c492c4e832298f85a9643d87ac7c06ba5aa0eba575901e890623e85b3 not found: ID does not exist"
Oct 30 01:54:11.924532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.924509    4577 scope.go:122] "RemoveContainer" containerID="52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"
Oct 30 01:54:11.924728 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:11.924711    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6\": container with ID starting with 52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6 not found: ID does not exist" containerID="52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"
Oct 30 01:54:11.924774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.924734    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6"} err="failed to get container status \"52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6\": rpc error: code = NotFound desc = could not find container \"52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6\": container with ID starting with 52b8a199121c7210d9e00fd2fe13cad7a672c26cb2a7a2e198e90f17720e75b6 not found: ID does not exist"
Oct 30 01:54:11.999867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:11.990739    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:12.012871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:12.009156    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"]
Oct 30 01:54:12.018460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:12.015278    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-bdf6ca-openshift-default-954f9568c-nchqq"]
Oct 30 01:54:12.114432 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:12.114394    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" path="/var/lib/kubelet/pods/92ddec96-952c-4a03-ae26-09aa2368fdfb/volumes"
Oct 30 01:54:13.805404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:13.805309    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:15.725178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:15.725146    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:16.873835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:16.873795    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:54:17.524655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:17.524612    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"]
Oct 30 01:54:17.546279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:17.546235    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:17.662930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:17.662891    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:17.789447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:17.789355    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:19.019387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:19.019345    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:19.683060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:19.683030    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:19.794822 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:19.794785    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:20.233491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:20.233460    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:21.211894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.211860    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"]
Oct 30 01:54:21.212256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.212173    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" podUID="fb114e8e-a391-43ca-aaae-bf52d6518b37" containerName="maas-api" containerID="cri-o://eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165" gracePeriod=30
Oct 30 01:54:21.304756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.304722    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:21.565871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.565422    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:54:21.725869 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.721554    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs\") pod \"fb114e8e-a391-43ca-aaae-bf52d6518b37\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") "
Oct 30 01:54:21.743888 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.733050    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs" pod "fb114e8e-a391-43ca-aaae-bf52d6518b37" (UID: "fb114e8e-a391-43ca-aaae-bf52d6518b37"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:21.768082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.746138    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\") pod \"fb114e8e-a391-43ca-aaae-bf52d6518b37\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") "
Oct 30 01:54:21.768082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.746260    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl\" (UniqueName: \"kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl\") pod \"fb114e8e-a391-43ca-aaae-bf52d6518b37\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") "
Oct 30 01:54:21.768082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.746317    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca\") pod \"fb114e8e-a391-43ca-aaae-bf52d6518b37\" (UID: \"fb114e8e-a391-43ca-aaae-bf52d6518b37\") "
Oct 30 01:54:21.768082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.761070    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca" pod "fb114e8e-a391-43ca-aaae-bf52d6518b37" (UID: "fb114e8e-a391-43ca-aaae-bf52d6518b37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:21.768506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.768384    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb114e8e-a391-43ca-aaae-bf52d6518b37-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:21.768506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.768407    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:21.783905 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.776805    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl" pod "fb114e8e-a391-43ca-aaae-bf52d6518b37" (UID: "fb114e8e-a391-43ca-aaae-bf52d6518b37"). InnerVolumeSpecName "kube-api-access-4f7zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:21.794437 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.791961    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls" pod "fb114e8e-a391-43ca-aaae-bf52d6518b37" (UID: "fb114e8e-a391-43ca-aaae-bf52d6518b37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:21.861957 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.861917    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:21.869353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.869332    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb114e8e-a391-43ca-aaae-bf52d6518b37-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:21.869438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.869355    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f7zl\" (UniqueName: \"kubernetes.io/projected/fb114e8e-a391-43ca-aaae-bf52d6518b37-kube-api-access-4f7zl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:21.979603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.979569    4577 generic.go:356] "Generic (PLEG): container finished" podID="fb114e8e-a391-43ca-aaae-bf52d6518b37" containerID="eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165" exitCode=0
Oct 30 01:54:21.979780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.979611    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"
Oct 30 01:54:21.979780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.979627    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" event={"ID":"fb114e8e-a391-43ca-aaae-bf52d6518b37","Type":"ContainerDied","Data":"eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165"}
Oct 30 01:54:21.979780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.979670    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp" event={"ID":"fb114e8e-a391-43ca-aaae-bf52d6518b37","Type":"ContainerDied","Data":"595ebd0f1755fe07c19437295c07f4c55c49dff0d80a46315f7589fe11b5965a"}
Oct 30 01:54:21.979780 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.979691    4577 scope.go:122] "RemoveContainer" containerID="eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165"
Oct 30 01:54:21.983408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.983386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:21.992603 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.992587    4577 scope.go:122] "RemoveContainer" containerID="eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165"
Oct 30 01:54:21.992913 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:21.992879    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165\": container with ID starting with eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165 not found: ID does not exist" containerID="eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165"
Oct 30 01:54:21.993039 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:21.992917    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165"} err="failed to get container status \"eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165\": rpc error: code = NotFound desc = could not find container \"eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165\": container with ID starting with eacc20049af41c6148714053ba005d5690fb1cee83f6346810aa94e5442c6165 not found: ID does not exist"
Oct 30 01:54:22.072989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:22.070896    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"]
Oct 30 01:54:22.080041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:22.079995    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-35042a-778598fcbc-7mmhp"]
Oct 30 01:54:22.109785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:22.109740    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:22.113136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:22.113113    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb114e8e-a391-43ca-aaae-bf52d6518b37" path="/var/lib/kubelet/pods/fb114e8e-a391-43ca-aaae-bf52d6518b37/volumes"
Oct 30 01:54:22.481191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:22.481157    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:22.525386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:22.525350    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" containerName="payload-processing" containerID="cri-o://765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b" gracePeriod=25
Oct 30 01:54:23.546503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:23.546470    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"]
Oct 30 01:54:23.547196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:23.547167    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" podUID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" containerName="maas-api" containerID="cri-o://41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3" gracePeriod=30
Oct 30 01:54:23.583769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:23.583648    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" podUID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.233:8443/health\": dial tcp 10.134.0.233:8443: connect: connection refused"
Oct 30 01:54:23.613454 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:23.613421    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:23.724137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:23.724105    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:23.900873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:23.897637    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:54:24.071954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.069144    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs\") pod \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") "
Oct 30 01:54:24.071954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.069209    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb\" (UniqueName: \"kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb\") pod \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") "
Oct 30 01:54:24.071954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.069390    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\") pod \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") "
Oct 30 01:54:24.071954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.069449    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca\") pod \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\" (UID: \"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53\") "
Oct 30 01:54:24.101769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.099655    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb" pod "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" (UID: "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53"). InnerVolumeSpecName "kube-api-access-l8xmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:24.112743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.109108    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca" pod "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" (UID: "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:24.112743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.109448    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls" pod "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" (UID: "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:24.121181 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.118872    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs" pod "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" (UID: "55ceaeb2-4b48-44d1-8a1b-b4ca43008a53"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:24.127139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.126168    4577 generic.go:356] "Generic (PLEG): container finished" podID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" containerID="41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3" exitCode=0
Oct 30 01:54:24.127139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.126481    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"
Oct 30 01:54:24.136353 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.136306    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" event={"ID":"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53","Type":"ContainerDied","Data":"41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3"}
Oct 30 01:54:24.136504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.136380    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx" event={"ID":"55ceaeb2-4b48-44d1-8a1b-b4ca43008a53","Type":"ContainerDied","Data":"5a49834136b7dc666e892eddb3c3d7c62f5927455ef705d451716059bb812d2c"}
Oct 30 01:54:24.136504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.136420    4577 scope.go:122] "RemoveContainer" containerID="41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3"
Oct 30 01:54:24.156200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.156178    4577 scope.go:122] "RemoveContainer" containerID="41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3"
Oct 30 01:54:24.156637 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:24.156618    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3\": container with ID starting with 41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3 not found: ID does not exist" containerID="41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3"
Oct 30 01:54:24.156714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.156651    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3"} err="failed to get container status \"41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3\": rpc error: code = NotFound desc = could not find container \"41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3\": container with ID starting with 41098895e816de74a5861313dec1d0ff8f205aee54eb574e141acb4b02f02de3 not found: ID does not exist"
Oct 30 01:54:24.171211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.171174    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:24.171211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.171210    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8xmb\" (UniqueName: \"kubernetes.io/projected/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-kube-api-access-l8xmb\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:24.171410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.171225    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:24.171410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.171238    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:24.187304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.186997    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"]
Oct 30 01:54:24.191723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.191697    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:24.195306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.195277    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-ae1820-779465cfcd-mknrx"]
Oct 30 01:54:24.312408 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.312366    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:24.440794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.440720    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:24.918494 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:24.918457    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:25.606515 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.606490    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:54:25.696709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.696669    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume\") pod \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") "
Oct 30 01:54:25.696937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.696923    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q\" (UniqueName: \"kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q\") pod \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\" (UID: \"421abe2b-d2f3-4455-81c9-1ac65acedd8f\") "
Oct 30 01:54:25.697120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.697097    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume" pod "421abe2b-d2f3-4455-81c9-1ac65acedd8f" (UID: "421abe2b-d2f3-4455-81c9-1ac65acedd8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:25.697346 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.697332    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/421abe2b-d2f3-4455-81c9-1ac65acedd8f-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:25.699529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.699504    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q" pod "421abe2b-d2f3-4455-81c9-1ac65acedd8f" (UID: "421abe2b-d2f3-4455-81c9-1ac65acedd8f"). InnerVolumeSpecName "kube-api-access-7qw7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:25.798338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:25.798304    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qw7q\" (UniqueName: \"kubernetes.io/projected/421abe2b-d2f3-4455-81c9-1ac65acedd8f-kube-api-access-7qw7q\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:26.111244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.111209    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" path="/var/lib/kubelet/pods/55ceaeb2-4b48-44d1-8a1b-b4ca43008a53/volumes"
Oct 30 01:54:26.143564 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.143530    4577 generic.go:356] "Generic (PLEG): container finished" podID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerID="c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28" exitCode=0
Oct 30 01:54:26.143765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.143590    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"
Oct 30 01:54:26.143765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.143608    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" event={"ID":"421abe2b-d2f3-4455-81c9-1ac65acedd8f","Type":"ContainerDied","Data":"c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28"}
Oct 30 01:54:26.143765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.143667    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2" event={"ID":"421abe2b-d2f3-4455-81c9-1ac65acedd8f","Type":"ContainerDied","Data":"4ea81f13cfa3a5b014f874bc7a784c3dcd17c16f9ab560da324dcb1b14863661"}
Oct 30 01:54:26.143765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.143692    4577 scope.go:122] "RemoveContainer" containerID="c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28"
Oct 30 01:54:26.154057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.154033    4577 scope.go:122] "RemoveContainer" containerID="c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28"
Oct 30 01:54:26.154372 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:26.154340    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28\": container with ID starting with c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28 not found: ID does not exist" containerID="c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28"
Oct 30 01:54:26.154448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.154384    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28"} err="failed to get container status \"c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28\": rpc error: code = NotFound desc = could not find container \"c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28\": container with ID starting with c5ce6820421f81d496cf70eb387e254b4ae684eb6b7ceb627bb388871ab1bf28 not found: ID does not exist"
Oct 30 01:54:26.201951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.201919    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"]
Oct 30 01:54:26.209061 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.209034    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-35042a-86ff6685d-dwwg2"]
Oct 30 01:54:26.334447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.334415    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:26.859759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:26.859719    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main" containerID="cri-o://5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6" gracePeriod=45
Oct 30 01:54:27.017710 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:27.017674    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:54:27.125320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.125282    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:27.138671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.138650    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 01:54:27.154352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.154321    4577 generic.go:356] "Generic (PLEG): container finished" podID="44192915-065f-4bca-820e-e901202dfb53" containerID="5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6" exitCode=0
Oct 30 01:54:27.154545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.154366    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"
Oct 30 01:54:27.154545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.154394    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" event={"ID":"44192915-065f-4bca-820e-e901202dfb53","Type":"ContainerDied","Data":"5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6"}
Oct 30 01:54:27.154545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.154449    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44" event={"ID":"44192915-065f-4bca-820e-e901202dfb53","Type":"ContainerDied","Data":"f8c024087340d52e762b7029fdc85fac47be4df608ae58b4efea01bab3343971"}
Oct 30 01:54:27.154545 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.154472    4577 scope.go:122] "RemoveContainer" containerID="5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6"
Oct 30 01:54:27.166403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.166383    4577 scope.go:122] "RemoveContainer" containerID="5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6"
Oct 30 01:54:27.166711 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:27.166694    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6\": container with ID starting with 5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6 not found: ID does not exist" containerID="5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6"
Oct 30 01:54:27.166793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.166721    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6"} err="failed to get container status \"5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6\": rpc error: code = NotFound desc = could not find container \"5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6\": container with ID starting with 5112853b46ae8bc59d303b6b99219feacd706cabd00c0003d63f623b4e958fb6 not found: ID does not exist"
Oct 30 01:54:27.255091 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.251386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:27.322644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322618    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home\") pod \"44192915-065f-4bca-820e-e901202dfb53\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") "
Oct 30 01:54:27.322832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322663    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d\" (UniqueName: \"kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d\") pod \"44192915-065f-4bca-820e-e901202dfb53\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") "
Oct 30 01:54:27.322832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322709    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache\") pod \"44192915-065f-4bca-820e-e901202dfb53\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") "
Oct 30 01:54:27.322832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322773    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs\" (UniqueName: \"kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs\") pod \"44192915-065f-4bca-820e-e901202dfb53\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") "
Oct 30 01:54:27.323045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322904    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir\") pod \"44192915-065f-4bca-820e-e901202dfb53\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") "
Oct 30 01:54:27.323045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322925    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home" pod "44192915-065f-4bca-820e-e901202dfb53" (UID: "44192915-065f-4bca-820e-e901202dfb53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:27.323045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.322963    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm\") pod \"44192915-065f-4bca-820e-e901202dfb53\" (UID: \"44192915-065f-4bca-820e-e901202dfb53\") "
Oct 30 01:54:27.323285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.323039    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache" pod "44192915-065f-4bca-820e-e901202dfb53" (UID: "44192915-065f-4bca-820e-e901202dfb53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:27.323285 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.323164    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir" pod "44192915-065f-4bca-820e-e901202dfb53" (UID: "44192915-065f-4bca-820e-e901202dfb53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:27.323599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.323535    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:27.323599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.323562    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:27.323599 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.323578    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:27.325216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.325187    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm" pod "44192915-065f-4bca-820e-e901202dfb53" (UID: "44192915-065f-4bca-820e-e901202dfb53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:27.325345 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.325254    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d" pod "44192915-065f-4bca-820e-e901202dfb53" (UID: "44192915-065f-4bca-820e-e901202dfb53"). InnerVolumeSpecName "kube-api-access-zn79d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:27.325487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.325464    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs" pod "44192915-065f-4bca-820e-e901202dfb53" (UID: "44192915-065f-4bca-820e-e901202dfb53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:27.363200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.363162    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:27.425197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.425120    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44192915-065f-4bca-820e-e901202dfb53-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:27.425197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.425149    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn79d\" (UniqueName: \"kubernetes.io/projected/44192915-065f-4bca-820e-e901202dfb53-kube-api-access-zn79d\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:27.425197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.425160    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44192915-065f-4bca-820e-e901202dfb53-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:27.482934 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.482894    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:27.507337 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.507312    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"]
Oct 30 01:54:27.513378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:27.513357    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-39b241/distinct-2-w3-39b241-kserve-6f5584d658-5hn44"]
Oct 30 01:54:28.112979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:28.112948    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" path="/var/lib/kubelet/pods/421abe2b-d2f3-4455-81c9-1ac65acedd8f/volumes"
Oct 30 01:54:28.113412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:28.113398    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44192915-065f-4bca-820e-e901202dfb53" path="/var/lib/kubelet/pods/44192915-065f-4bca-820e-e901202dfb53/volumes"
Oct 30 01:54:28.228119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:28.228083    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:30.015232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:30.015192    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:30.202541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:30.202504    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:30.339253 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:30.339132    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:30.340218 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:30.340191    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"]
Oct 30 01:54:30.340455 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:30.340430    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy" containerID="cri-o://585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9" gracePeriod=30
Oct 30 01:54:32.100556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:32.100518    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:32.431624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:32.431529    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:54:33.116281 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:33.116251    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:33.278929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:33.276923    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:33.465913 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:33.465823    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:33.989293 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:33.989241    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:35.831661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:35.831623    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:36.065881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.065834    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:54:36.222497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222405    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.222497 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222467    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.222744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222505    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.222744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222539    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.222744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222616    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.222744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222676    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.222744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222734    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.223063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222781    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.223063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222868    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9\") pod \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\" (UID: \"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d\") "
Oct 30 01:54:36.223063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222953    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:36.223063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.222984    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:36.223063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223022    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:36.223063 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223030    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:36.223563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223195    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:36.223563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223507    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.223563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223527    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.223563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223542    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.223563 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223556    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.223866 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.223570    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.225009 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.224987    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:54:36.225124 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.225072    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:36.225573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.225551    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:54:36.225685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.225630    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9" pod "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" (UID: "6a1e68a2-1048-47ae-a00e-3fa2f63fa45d"). InnerVolumeSpecName "kube-api-access-b4nb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:36.261183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.261152    4577 generic.go:356] "Generic (PLEG): container finished" podID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerID="585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9" exitCode=0
Oct 30 01:54:36.261320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.261203    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"
Oct 30 01:54:36.261320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.261240    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" event={"ID":"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d","Type":"ContainerDied","Data":"585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9"}
Oct 30 01:54:36.261320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.261307    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r" event={"ID":"6a1e68a2-1048-47ae-a00e-3fa2f63fa45d","Type":"ContainerDied","Data":"e734ac2ae389a567cbe7bc5e1b703871fa5de5287774f65e4fba55d566b2e54b"}
Oct 30 01:54:36.261446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.261347    4577 scope.go:122] "RemoveContainer" containerID="585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9"
Oct 30 01:54:36.274322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.274296    4577 scope.go:122] "RemoveContainer" containerID="2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"
Oct 30 01:54:36.284670 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.284648    4577 scope.go:122] "RemoveContainer" containerID="585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9"
Oct 30 01:54:36.284985 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:36.284960    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9\": container with ID starting with 585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9 not found: ID does not exist" containerID="585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9"
Oct 30 01:54:36.285077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.284997    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9"} err="failed to get container status \"585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9\": rpc error: code = NotFound desc = could not find container \"585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9\": container with ID starting with 585b588dd276e5c713e73201fd7f594f343d352374aed64f090d6044f82583d9 not found: ID does not exist"
Oct 30 01:54:36.285077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.285022    4577 scope.go:122] "RemoveContainer" containerID="2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"
Oct 30 01:54:36.285374 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:36.285345    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9\": container with ID starting with 2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9 not found: ID does not exist" containerID="2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"
Oct 30 01:54:36.285438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.285382    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9"} err="failed to get container status \"2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9\": rpc error: code = NotFound desc = could not find container \"2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9\": container with ID starting with 2ab6c4aa06611f74ffec8ab1974766033d63e06247ec36b656c83d09ce9b3de9 not found: ID does not exist"
Oct 30 01:54:36.320105 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.320075    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"]
Oct 30 01:54:36.325516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.325488    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.325654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.325522    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4nb9\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-kube-api-access-b4nb9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.325654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.325538    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.325654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.325551    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:36.330460 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:36.330438    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-35042a-openshift-default-566d5df757-l6b6r"]
Oct 30 01:54:37.604674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:37.604645    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:38.112055 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:38.112013    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" path="/var/lib/kubelet/pods/6a1e68a2-1048-47ae-a00e-3fa2f63fa45d/volumes"
Oct 30 01:54:38.672068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:38.672024    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:38.803673 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:38.803634    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:39.277607 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:39.277576    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:41.097377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:41.097341    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:41.484516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:41.484483    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"]
Oct 30 01:54:41.607551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:41.607516    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:41.728415 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:41.728377    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:42.147601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.147566    4577 scope.go:122] "RemoveContainer" containerID="2c4a5ebea899c1fc6c219385d1f62560fe692da78e809b5f71e0828ce15d3b88"
Oct 30 01:54:42.431551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.431460    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:54:42.803441 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.803417    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 01:54:42.845287 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.845241    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume\") pod \"cf82a9c9-e781-431f-a03b-b0792078523c\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") "
Oct 30 01:54:42.845547 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.845335    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv\" (UniqueName: \"kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv\") pod \"cf82a9c9-e781-431f-a03b-b0792078523c\" (UID: \"cf82a9c9-e781-431f-a03b-b0792078523c\") "
Oct 30 01:54:42.846242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.846201    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume" pod "cf82a9c9-e781-431f-a03b-b0792078523c" (UID: "cf82a9c9-e781-431f-a03b-b0792078523c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:42.855308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.855269    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv" pod "cf82a9c9-e781-431f-a03b-b0792078523c" (UID: "cf82a9c9-e781-431f-a03b-b0792078523c"). InnerVolumeSpecName "kube-api-access-jhhzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:42.915119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.915086    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:42.947809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.947769    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf82a9c9-e781-431f-a03b-b0792078523c-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:42.947809 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:42.947798    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhhzv\" (UniqueName: \"kubernetes.io/projected/cf82a9c9-e781-431f-a03b-b0792078523c-kube-api-access-jhhzv\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:43.316590 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.316557    4577 generic.go:356] "Generic (PLEG): container finished" podID="cf82a9c9-e781-431f-a03b-b0792078523c" containerID="765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b" exitCode=0
Oct 30 01:54:43.317068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.316607    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"
Oct 30 01:54:43.317068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.316643    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" event={"ID":"cf82a9c9-e781-431f-a03b-b0792078523c","Type":"ContainerDied","Data":"765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b"}
Oct 30 01:54:43.317068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.316688    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8" event={"ID":"cf82a9c9-e781-431f-a03b-b0792078523c","Type":"ContainerDied","Data":"70b0e810064ca531eb37e1dc2df878f6afa3fec12d372bf24de9ec36ce1ee099"}
Oct 30 01:54:43.317068 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.316711    4577 scope.go:122] "RemoveContainer" containerID="765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b"
Oct 30 01:54:43.331553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.331223    4577 scope.go:122] "RemoveContainer" containerID="765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b"
Oct 30 01:54:43.331650 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:43.331620    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b\": container with ID starting with 765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b not found: ID does not exist" containerID="765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b"
Oct 30 01:54:43.331740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.331661    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b"} err="failed to get container status \"765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b\": rpc error: code = NotFound desc = could not find container \"765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b\": container with ID starting with 765a0046bb5811e0502eea2af7fbec82916aa39a6d77432e580847c0bdb7982b not found: ID does not exist"
Oct 30 01:54:43.376469 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.376434    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"]
Oct 30 01:54:43.379526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:43.379502    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-39b241-86d56d7bf8-l8cf8"]
Oct 30 01:54:44.112635 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:44.112594    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" path="/var/lib/kubelet/pods/cf82a9c9-e781-431f-a03b-b0792078523c/volumes"
Oct 30 01:54:44.537744 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:44.537706    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:44.676507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:44.674481    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:46.233120 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:46.233083    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:46.484985 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:46.484892    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" containerName="payload-pre-processing" containerID="cri-o://6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac" gracePeriod=25
Oct 30 01:54:48.033140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:48.033106    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:48.445449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:48.445356    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:54:49.634385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:49.634346    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:49.812660 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:49.812616    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:50.009567 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:50.009420    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:51.625776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:51.625748    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:51.746553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:51.746520    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:52.131640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:52.131604    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:52.248014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:52.247981    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:52.365239 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:52.364467    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:52.795140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:52.795109    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:54.189115 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:54.189079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:55.702447 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:55.702410    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"]
Oct 30 01:54:55.702976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:55.702723    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz" podUID="54abb8e7-cc15-4828-970b-3b0493136625" containerName="maas-api" containerID="cri-o://b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7" gracePeriod=30
Oct 30 01:54:55.765198 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:55.765153    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:56.062396 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.060784    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:54:56.169883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.167536    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v\" (UniqueName: \"kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v\") pod \"54abb8e7-cc15-4828-970b-3b0493136625\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") "
Oct 30 01:54:56.169883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.167709    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs\") pod \"54abb8e7-cc15-4828-970b-3b0493136625\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") "
Oct 30 01:54:56.169883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.167821    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls\") pod \"54abb8e7-cc15-4828-970b-3b0493136625\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") "
Oct 30 01:54:56.169883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.167878    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca\") pod \"54abb8e7-cc15-4828-970b-3b0493136625\" (UID: \"54abb8e7-cc15-4828-970b-3b0493136625\") "
Oct 30 01:54:56.169883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.169264    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca" pod "54abb8e7-cc15-4828-970b-3b0493136625" (UID: "54abb8e7-cc15-4828-970b-3b0493136625"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:56.195883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.185266    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls" pod "54abb8e7-cc15-4828-970b-3b0493136625" (UID: "54abb8e7-cc15-4828-970b-3b0493136625"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:56.195883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.191043    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs" pod "54abb8e7-cc15-4828-970b-3b0493136625" (UID: "54abb8e7-cc15-4828-970b-3b0493136625"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:56.195883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.192025    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v" pod "54abb8e7-cc15-4828-970b-3b0493136625" (UID: "54abb8e7-cc15-4828-970b-3b0493136625"). InnerVolumeSpecName "kube-api-access-hhc4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:56.269976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.269881    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:56.269976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.269920    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54abb8e7-cc15-4828-970b-3b0493136625-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:56.269976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.269934    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54abb8e7-cc15-4828-970b-3b0493136625-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:56.269976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.269951    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhc4v\" (UniqueName: \"kubernetes.io/projected/54abb8e7-cc15-4828-970b-3b0493136625-kube-api-access-hhc4v\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:56.292312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.292277    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:56.406159 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.406128    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:56.418909 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.418875    4577 generic.go:356] "Generic (PLEG): container finished" podID="54abb8e7-cc15-4828-970b-3b0493136625" containerID="b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7" exitCode=0
Oct 30 01:54:56.419104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.418923    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"
Oct 30 01:54:56.419104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.418948    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz" event={"ID":"54abb8e7-cc15-4828-970b-3b0493136625","Type":"ContainerDied","Data":"b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7"}
Oct 30 01:54:56.419104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.419011    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz" event={"ID":"54abb8e7-cc15-4828-970b-3b0493136625","Type":"ContainerDied","Data":"52d2025c3e3012afb10a3f0adc87559dc8eecbca394bec7563062b99f1dd0408"}
Oct 30 01:54:56.419104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.419033    4577 scope.go:122] "RemoveContainer" containerID="b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7"
Oct 30 01:54:56.430308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.430288    4577 scope.go:122] "RemoveContainer" containerID="b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7"
Oct 30 01:54:56.430603 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:56.430583    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7\": container with ID starting with b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7 not found: ID does not exist" containerID="b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7"
Oct 30 01:54:56.430666 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.430612    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7"} err="failed to get container status \"b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7\": rpc error: code = NotFound desc = could not find container \"b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7\": container with ID starting with b12ca5349b4d1a7eeea4393fa9aca4cf2f7142c6cc5b18f2160a05a9a263fee7 not found: ID does not exist"
Oct 30 01:54:56.491006 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.490971    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"]
Oct 30 01:54:56.494191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.494164    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-1854c4-64d9c7ff75-87ckz"]
Oct 30 01:54:56.532125 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.532009    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:56.931826 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:56.931750    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:57.015005 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:57.014969    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:54:58.079706 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.079598    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:58.111605 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.111574    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54abb8e7-cc15-4828-970b-3b0493136625" path="/var/lib/kubelet/pods/54abb8e7-cc15-4828-970b-3b0493136625/volumes"
Oct 30 01:54:58.405088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.405003    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"]
Oct 30 01:54:58.405283 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.405217    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj" podUID="142ed92b-5389-4bff-96ba-683474be650f" containerName="maas-api" containerID="cri-o://a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a" gracePeriod=30
Oct 30 01:54:58.448257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.444891    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:54:58.467736 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:58.467687    4577 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142ed92b_5389_4bff_96ba_683474be650f.slice/crio-a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142ed92b_5389_4bff_96ba_683474be650f.slice/crio-conmon-a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 01:54:58.481668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.481601    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:58.605158 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.605116    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:58.698137 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.698103    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:54:58.718416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.718380    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:58.819837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.819802    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm\" (UniqueName: \"kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm\") pod \"142ed92b-5389-4bff-96ba-683474be650f\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") "
Oct 30 01:54:58.819837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.819866    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls\") pod \"142ed92b-5389-4bff-96ba-683474be650f\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") "
Oct 30 01:54:58.820100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.820019    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs\") pod \"142ed92b-5389-4bff-96ba-683474be650f\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") "
Oct 30 01:54:58.820100 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.820043    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca\") pod \"142ed92b-5389-4bff-96ba-683474be650f\" (UID: \"142ed92b-5389-4bff-96ba-683474be650f\") "
Oct 30 01:54:58.820403 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.820377    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca" pod "142ed92b-5389-4bff-96ba-683474be650f" (UID: "142ed92b-5389-4bff-96ba-683474be650f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:54:58.820834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.820813    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/142ed92b-5389-4bff-96ba-683474be650f-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:58.822294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.822264    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs" pod "142ed92b-5389-4bff-96ba-683474be650f" (UID: "142ed92b-5389-4bff-96ba-683474be650f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:58.822389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.822301    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls" pod "142ed92b-5389-4bff-96ba-683474be650f" (UID: "142ed92b-5389-4bff-96ba-683474be650f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:54:58.822389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.822330    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm" pod "142ed92b-5389-4bff-96ba-683474be650f" (UID: "142ed92b-5389-4bff-96ba-683474be650f"). InnerVolumeSpecName "kube-api-access-6s7lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:54:58.927926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.922825    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s7lm\" (UniqueName: \"kubernetes.io/projected/142ed92b-5389-4bff-96ba-683474be650f-kube-api-access-6s7lm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:58.927926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.922886    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:58.927926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.922901    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/142ed92b-5389-4bff-96ba-683474be650f-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:54:58.975870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:58.974970    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:59.087444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.087050    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:59.200094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.200010    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:59.323377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.323339    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:54:59.444612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.444569    4577 generic.go:356] "Generic (PLEG): container finished" podID="142ed92b-5389-4bff-96ba-683474be650f" containerID="a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a" exitCode=0
Oct 30 01:54:59.444612 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.444616    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"
Oct 30 01:54:59.444876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.444651    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj" event={"ID":"142ed92b-5389-4bff-96ba-683474be650f","Type":"ContainerDied","Data":"a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a"}
Oct 30 01:54:59.444876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.444708    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj" event={"ID":"142ed92b-5389-4bff-96ba-683474be650f","Type":"ContainerDied","Data":"d16c910b580fb5c9f63aa9bfa475ad6ef1e399840f3145df86f4f58252461513"}
Oct 30 01:54:59.444876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.444733    4577 scope.go:122] "RemoveContainer" containerID="a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a"
Oct 30 01:54:59.457573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.457549    4577 scope.go:122] "RemoveContainer" containerID="a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a"
Oct 30 01:54:59.457939 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:54:59.457917    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a\": container with ID starting with a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a not found: ID does not exist" containerID="a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a"
Oct 30 01:54:59.458025 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.457948    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a"} err="failed to get container status \"a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a\": rpc error: code = NotFound desc = could not find container \"a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a\": container with ID starting with a3a0128a2110c669e137d9168b9a246cb9e54fd3514d38a33076109449209c4a not found: ID does not exist"
Oct 30 01:54:59.507205 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.507166    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"]
Oct 30 01:54:59.517874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.517827    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-8ae41e-64456ff86d-kbmzj"]
Oct 30 01:54:59.731668 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:54:59.731634    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:00.114732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:00.114655    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="142ed92b-5389-4bff-96ba-683474be650f" path="/var/lib/kubelet/pods/142ed92b-5389-4bff-96ba-683474be650f/volumes"
Oct 30 01:55:00.499286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:00.499249    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:00.624266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:00.624235    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:00.797788 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:00.797690    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"]
Oct 30 01:55:00.797989 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:00.797916    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" containerID="cri-o://40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1" gracePeriod=30
Oct 30 01:55:00.831753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:00.831713    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:01.028586 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:01.028544    4577 patch_prober.go:29] interesting pod/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:55:01.028773 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:01.028603    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:55:01.263885 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:01.263774    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:03.105169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:03.105139    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:04.652182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:04.652139    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:04.774575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:04.774517    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:04.911810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:04.911716    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:05.564587 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:05.564544    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:05.711245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:05.711212    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:05.909530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:05.909498    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:06.206326 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.206300    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 01:55:06.338033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.337997    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338090    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338123    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338154    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338209    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338252    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338281    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338329    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.338579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.338366    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token\") pod \"f7b4a993-158d-453e-8818-aad628b254e9\" (UID: \"f7b4a993-158d-453e-8818-aad628b254e9\") "
Oct 30 01:55:06.339265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.339235    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:06.339548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.339419    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:06.339548 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.339490    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:06.339815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.339642    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:06.339815 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.339796    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:06.341581 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.341549    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:55:06.341701 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.341663    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:06.342621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.342598    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:06.343212 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.343191    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln" pod "f7b4a993-158d-453e-8818-aad628b254e9" (UID: "f7b4a993-158d-453e-8818-aad628b254e9"). InnerVolumeSpecName "kube-api-access-qrlln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:06.389966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.389938    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440403    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440438    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b4a993-158d-453e-8818-aad628b254e9-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440450    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440460    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440469    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440478    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b4a993-158d-453e-8818-aad628b254e9-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440487    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b4a993-158d-453e-8818-aad628b254e9-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440495    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrlln\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-kube-api-access-qrlln\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.440556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.440505    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b4a993-158d-453e-8818-aad628b254e9-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.505236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.505205    4577 generic.go:356] "Generic (PLEG): container finished" podID="f7b4a993-158d-453e-8818-aad628b254e9" containerID="40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1" exitCode=0
Oct 30 01:55:06.505417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.505262    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"
Oct 30 01:55:06.505417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.505291    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" event={"ID":"f7b4a993-158d-453e-8818-aad628b254e9","Type":"ContainerDied","Data":"40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1"}
Oct 30 01:55:06.505417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.505365    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm" event={"ID":"f7b4a993-158d-453e-8818-aad628b254e9","Type":"ContainerDied","Data":"881b889d591c438ae18830d60bb93505fe409b9f6a9fe3b539b5e5b2a8099b4f"}
Oct 30 01:55:06.505417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.505388    4577 scope.go:122] "RemoveContainer" containerID="40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1"
Oct 30 01:55:06.599477 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.599453    4577 scope.go:122] "RemoveContainer" containerID="40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1"
Oct 30 01:55:06.600038 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:06.600015    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1\": container with ID starting with 40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1 not found: ID does not exist" containerID="40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1"
Oct 30 01:55:06.600134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.600045    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1"} err="failed to get container status \"40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1\": rpc error: code = NotFound desc = could not find container \"40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1\": container with ID starting with 40c4ddf382c5bb64aa29c75631a29a60096c5c6a83f64ba7d1a5680c61ab43a1 not found: ID does not exist"
Oct 30 01:55:06.608906 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.608883    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"]
Oct 30 01:55:06.622874 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.622831    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-39b241-openshift-default-6bd9c8d777-48rvm"]
Oct 30 01:55:06.764827 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.764805    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:55:06.846358 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.846333    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7\" (UniqueName: \"kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7\") pod \"9a423130-f279-40f4-86fb-6120109a2a0b\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") "
Oct 30 01:55:06.846523 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.846483    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume\") pod \"9a423130-f279-40f4-86fb-6120109a2a0b\" (UID: \"9a423130-f279-40f4-86fb-6120109a2a0b\") "
Oct 30 01:55:06.847310 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.847281    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume" pod "9a423130-f279-40f4-86fb-6120109a2a0b" (UID: "9a423130-f279-40f4-86fb-6120109a2a0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:06.850256 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.850233    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7" pod "9a423130-f279-40f4-86fb-6120109a2a0b" (UID: "9a423130-f279-40f4-86fb-6120109a2a0b"). InnerVolumeSpecName "kube-api-access-bxdd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:06.947978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.947943    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a423130-f279-40f4-86fb-6120109a2a0b-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:06.947978 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:06.947977    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxdd7\" (UniqueName: \"kubernetes.io/projected/9a423130-f279-40f4-86fb-6120109a2a0b-kube-api-access-bxdd7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:07.143723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.143636    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:07.268241 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.268204    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:07.395261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.395160    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:07.513750 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.513714    4577 generic.go:356] "Generic (PLEG): container finished" podID="9a423130-f279-40f4-86fb-6120109a2a0b" containerID="6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac" exitCode=0
Oct 30 01:55:07.513968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.513757    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" event={"ID":"9a423130-f279-40f4-86fb-6120109a2a0b","Type":"ContainerDied","Data":"6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac"}
Oct 30 01:55:07.513968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.513766    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"
Oct 30 01:55:07.513968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.513810    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj" event={"ID":"9a423130-f279-40f4-86fb-6120109a2a0b","Type":"ContainerDied","Data":"6272c776e8e27bafa1280a50a200e8ab1079e428ebb17f00a95cd2ebda27682f"}
Oct 30 01:55:07.513968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.513857    4577 scope.go:122] "RemoveContainer" containerID="6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac"
Oct 30 01:55:07.524578 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.524561    4577 scope.go:122] "RemoveContainer" containerID="6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac"
Oct 30 01:55:07.524914 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:07.524896    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac\": container with ID starting with 6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac not found: ID does not exist" containerID="6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac"
Oct 30 01:55:07.524984 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.524922    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac"} err="failed to get container status \"6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac\": rpc error: code = NotFound desc = could not find container \"6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac\": container with ID starting with 6d0ff137f9fd3f95f10523dc605e3233078a21f5f4e7365340432aac0eb6eeac not found: ID does not exist"
Oct 30 01:55:07.585941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.585895    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"]
Oct 30 01:55:07.597982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.597919    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9da49f-6f9f859f8c-958dj"]
Oct 30 01:55:07.608891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.602102    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"]
Oct 30 01:55:07.608891 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.602421    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy" containerID="cri-o://91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22" gracePeriod=30
Oct 30 01:55:07.660324 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:07.656404    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:08.066288 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.066243    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:08.111883 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.111822    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" path="/var/lib/kubelet/pods/9a423130-f279-40f4-86fb-6120109a2a0b/volumes"
Oct 30 01:55:08.112352 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.112336    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7b4a993-158d-453e-8818-aad628b254e9" path="/var/lib/kubelet/pods/f7b4a993-158d-453e-8818-aad628b254e9/volumes"
Oct 30 01:55:08.321980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.321830    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:08.459758 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.459710    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:08.668464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.668170    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"]
Oct 30 01:55:08.668464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.668435    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" containerID="cri-o://38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4" gracePeriod=30
Oct 30 01:55:08.701101 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:08.701071    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:09.293746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:09.293714    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:09.665079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:09.664998    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:55:09.665079 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:09.665054    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:55:11.231054 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:11.231024    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:11.367234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:11.367201    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"]
Oct 30 01:55:11.466097 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:11.466060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:12.875674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.875605    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:55:12.955419 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955377    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955438    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955515    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955543    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955598    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955635    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955756    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.955893 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955792    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.956069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.955893    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy\") pod \"9998a2ee-6eba-4438-8713-af86376d2dc2\" (UID: \"9998a2ee-6eba-4438-8713-af86376d2dc2\") "
Oct 30 01:55:12.956231 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.956200    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:12.956550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.956524    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9998a2ee-6eba-4438-8713-af86376d2dc2-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:12.956933 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.956909    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:12.957041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.956945    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:12.957041 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.957025    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:12.957203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.957083    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:12.965900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.959230    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "kube-api-access-f8m82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:12.965900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.959519    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:12.965900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.959576    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:55:12.965900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:12.959893    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token" pod "9998a2ee-6eba-4438-8713-af86376d2dc2" (UID: "9998a2ee-6eba-4438-8713-af86376d2dc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:13.000468 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.000435    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:13.058161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058125    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058164    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058177    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058190    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058203    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058220    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8m82\" (UniqueName: \"kubernetes.io/projected/9998a2ee-6eba-4438-8713-af86376d2dc2-kube-api-access-f8m82\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058233    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.058372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.058248    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9998a2ee-6eba-4438-8713-af86376d2dc2-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:13.153707 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.153586    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:13.560242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.560209    4577 generic.go:356] "Generic (PLEG): container finished" podID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerID="91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22" exitCode=0
Oct 30 01:55:13.560242 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.560250    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" event={"ID":"9998a2ee-6eba-4438-8713-af86376d2dc2","Type":"ContainerDied","Data":"91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22"}
Oct 30 01:55:13.560501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.560288    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"
Oct 30 01:55:13.560501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.560298    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq" event={"ID":"9998a2ee-6eba-4438-8713-af86376d2dc2","Type":"ContainerDied","Data":"3cffeb19c3c052b9f89b324f311dd1d9fed51e7536773212c48929f01d3479d2"}
Oct 30 01:55:13.560501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.560331    4577 scope.go:122] "RemoveContainer" containerID="91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22"
Oct 30 01:55:13.571211 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.571192    4577 scope.go:122] "RemoveContainer" containerID="bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"
Oct 30 01:55:13.580634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.580613    4577 scope.go:122] "RemoveContainer" containerID="91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22"
Oct 30 01:55:13.580934 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:13.580914    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22\": container with ID starting with 91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22 not found: ID does not exist" containerID="91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22"
Oct 30 01:55:13.581017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.580941    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22"} err="failed to get container status \"91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22\": rpc error: code = NotFound desc = could not find container \"91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22\": container with ID starting with 91f2246f6358c7d5cc2e326b713e09317a727135eda9feb5f16af9b8f6bd0d22 not found: ID does not exist"
Oct 30 01:55:13.581017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.580958    4577 scope.go:122] "RemoveContainer" containerID="bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"
Oct 30 01:55:13.581236 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:13.581214    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3\": container with ID starting with bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3 not found: ID does not exist" containerID="bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"
Oct 30 01:55:13.581354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.581246    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3"} err="failed to get container status \"bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3\": rpc error: code = NotFound desc = could not find container \"bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3\": container with ID starting with bd8128ed1048d9b66ac41c852383ff33d7329dc73ada7bfbd0528e96014562b3 not found: ID does not exist"
Oct 30 01:55:13.634540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.634511    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"]
Oct 30 01:55:13.648444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.648415    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9da49f-openshift-default-556d9c56f5-vhlvq"]
Oct 30 01:55:13.944871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:13.944829    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:55:14.078363 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078329    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078459    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078490    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078513    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078564    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078601    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078640    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078688    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.078811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078741    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert\") pod \"38c3a078-0da3-44e6-91b9-32604ce48f3d\" (UID: \"38c3a078-0da3-44e6-91b9-32604ce48f3d\") "
Oct 30 01:55:14.079095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078981    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:14.079095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.078995    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:14.079095 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079073    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:14.079354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079303    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:14.079354 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079325    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:14.079498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079481    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.079557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079505    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38c3a078-0da3-44e6-91b9-32604ce48f3d-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.079557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079523    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.079557 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079537    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.079671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.079553    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.080881 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.080824    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:14.081226 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.081206    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:14.081389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.081371    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:55:14.081389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.081380    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv" pod "38c3a078-0da3-44e6-91b9-32604ce48f3d" (UID: "38c3a078-0da3-44e6-91b9-32604ce48f3d"). InnerVolumeSpecName "kube-api-access-9jdkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:14.111835 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.111744    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" path="/var/lib/kubelet/pods/9998a2ee-6eba-4438-8713-af86376d2dc2/volumes"
Oct 30 01:55:14.180951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.180918    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.180951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.180946    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.180951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.180956    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38c3a078-0da3-44e6-91b9-32604ce48f3d-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.181203 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.180965    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/38c3a078-0da3-44e6-91b9-32604ce48f3d-kube-api-access-9jdkv\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:14.571385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.571350    4577 generic.go:356] "Generic (PLEG): container finished" podID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerID="38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4" exitCode=0
Oct 30 01:55:14.571642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.571403    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"
Oct 30 01:55:14.571642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.571431    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" event={"ID":"38c3a078-0da3-44e6-91b9-32604ce48f3d","Type":"ContainerDied","Data":"38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4"}
Oct 30 01:55:14.571642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.571487    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8" event={"ID":"38c3a078-0da3-44e6-91b9-32604ce48f3d","Type":"ContainerDied","Data":"7050ae6b7b01682e4243f4782aa6d313890cacae48bfe02da09c6fc808afb99a"}
Oct 30 01:55:14.571642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.571510    4577 scope.go:122] "RemoveContainer" containerID="38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4"
Oct 30 01:55:14.582723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.582699    4577 scope.go:122] "RemoveContainer" containerID="ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"
Oct 30 01:55:14.593962 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.593937    4577 scope.go:122] "RemoveContainer" containerID="38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4"
Oct 30 01:55:14.594306 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:14.594287    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4\": container with ID starting with 38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4 not found: ID does not exist" containerID="38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4"
Oct 30 01:55:14.594417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.594323    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4"} err="failed to get container status \"38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4\": rpc error: code = NotFound desc = could not find container \"38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4\": container with ID starting with 38d97101e37fa1f866ca6306d23eba5de1d5cfe3861d2678d27584d4cf7305e4 not found: ID does not exist"
Oct 30 01:55:14.594417 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.594347    4577 scope.go:122] "RemoveContainer" containerID="ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"
Oct 30 01:55:14.594687 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:14.594664    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933\": container with ID starting with ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933 not found: ID does not exist" containerID="ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"
Oct 30 01:55:14.594743 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.594694    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933"} err="failed to get container status \"ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933\": rpc error: code = NotFound desc = could not find container \"ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933\": container with ID starting with ea40395b5bdf2034428ad09d76776379efe4505cd9fac89d2e2afd0f468ea933 not found: ID does not exist"
Oct 30 01:55:14.631096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.631058    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"]
Oct 30 01:55:14.644654 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.644621    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-8ae41e-openshift-default-6775d5944c-89qc8"]
Oct 30 01:55:14.815167 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:14.815127    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:16.111387 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:16.111350    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" path="/var/lib/kubelet/pods/38c3a078-0da3-44e6-91b9-32604ce48f3d/volumes"
Oct 30 01:55:16.461129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:16.461026    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:17.058792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:17.058735    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"]
Oct 30 01:55:17.075683 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:17.075644    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"]
Oct 30 01:55:17.369685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:17.369565    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:17.491721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:17.491693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:18.416448 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:18.416362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:18.724753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:18.724656    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"]
Oct 30 01:55:18.738251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:18.738218    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"]
Oct 30 01:55:18.848416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:18.848373    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:19.049179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:19.049126    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:19.615552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:19.615523    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:19.732610 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:19.732358    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:20.329342 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:20.329304    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:21.992116 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:21.992079    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:22.059739 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:22.059636    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerName="payload-processing" containerID="cri-o://375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314" gracePeriod=25
Oct 30 01:55:22.076094 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:22.075997    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerName="payload-pre-processing" containerID="cri-o://38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c" gracePeriod=25
Oct 30 01:55:23.479007 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.478973    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"]
Oct 30 01:55:23.479485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.479201    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg" podUID="f852d0d3-9901-48da-af1f-f292a7731a96" containerName="maas-api" containerID="cri-o://ed211d098f722c2f4de17af9af10a15441c034c9c29bd375c39553a6c7620844" gracePeriod=30
Oct 30 01:55:23.548258 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.548227    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:23.653166 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.653120    4577 generic.go:356] "Generic (PLEG): container finished" podID="f852d0d3-9901-48da-af1f-f292a7731a96" containerID="ed211d098f722c2f4de17af9af10a15441c034c9c29bd375c39553a6c7620844" exitCode=0
Oct 30 01:55:23.653344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.653303    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg" event={"ID":"f852d0d3-9901-48da-af1f-f292a7731a96","Type":"ContainerDied","Data":"ed211d098f722c2f4de17af9af10a15441c034c9c29bd375c39553a6c7620844"}
Oct 30 01:55:23.738928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.728946    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h" podUID="fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" containerName="payload-processing" containerID="cri-o://458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37" gracePeriod=25
Oct 30 01:55:23.760868 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.754163    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs" podUID="c41e5112-952a-4a1d-841b-be3d0900b6dd" containerName="payload-pre-processing" containerID="cri-o://0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b" gracePeriod=25
Oct 30 01:55:23.954574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:23.954541    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:55:24.025649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.023626    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls\") pod \"f852d0d3-9901-48da-af1f-f292a7731a96\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") "
Oct 30 01:55:24.025649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.023727    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs\") pod \"f852d0d3-9901-48da-af1f-f292a7731a96\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") "
Oct 30 01:55:24.025649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.023923    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca\") pod \"f852d0d3-9901-48da-af1f-f292a7731a96\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") "
Oct 30 01:55:24.025649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.023971    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4\" (UniqueName: \"kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4\") pod \"f852d0d3-9901-48da-af1f-f292a7731a96\" (UID: \"f852d0d3-9901-48da-af1f-f292a7731a96\") "
Oct 30 01:55:24.025649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.024805    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca" pod "f852d0d3-9901-48da-af1f-f292a7731a96" (UID: "f852d0d3-9901-48da-af1f-f292a7731a96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:24.043490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.038258    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4" pod "f852d0d3-9901-48da-af1f-f292a7731a96" (UID: "f852d0d3-9901-48da-af1f-f292a7731a96"). InnerVolumeSpecName "kube-api-access-pvbf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:24.043490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.038351    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls" pod "f852d0d3-9901-48da-af1f-f292a7731a96" (UID: "f852d0d3-9901-48da-af1f-f292a7731a96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:24.043490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.040374    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs" pod "f852d0d3-9901-48da-af1f-f292a7731a96" (UID: "f852d0d3-9901-48da-af1f-f292a7731a96"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:24.088106 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.087733    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:24.126316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.126285    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f852d0d3-9901-48da-af1f-f292a7731a96-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.126316 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.126318    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvbf4\" (UniqueName: \"kubernetes.io/projected/f852d0d3-9901-48da-af1f-f292a7731a96-kube-api-access-pvbf4\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.126532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.126333    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.126532 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.126342    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f852d0d3-9901-48da-af1f-f292a7731a96-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.188309 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.188276    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 01:55:24.205506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.205470    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:24.209975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.209935    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 01:55:24.227327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.227291    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume\") pod \"c41e5112-952a-4a1d-841b-be3d0900b6dd\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") "
Oct 30 01:55:24.227511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.227476    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume\") pod \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") "
Oct 30 01:55:24.227511 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.227507    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd\" (UniqueName: \"kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd\") pod \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\" (UID: \"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910\") "
Oct 30 01:55:24.227662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.227589    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h\" (UniqueName: \"kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h\") pod \"c41e5112-952a-4a1d-841b-be3d0900b6dd\" (UID: \"c41e5112-952a-4a1d-841b-be3d0900b6dd\") "
Oct 30 01:55:24.228001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.227973    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume" pod "fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" (UID: "fc273ca8-5ef5-4d77-b4aa-483a1bd9d910"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:24.228169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.228130    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume" pod "c41e5112-952a-4a1d-841b-be3d0900b6dd" (UID: "c41e5112-952a-4a1d-841b-be3d0900b6dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:24.230917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.230889    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h" pod "c41e5112-952a-4a1d-841b-be3d0900b6dd" (UID: "c41e5112-952a-4a1d-841b-be3d0900b6dd"). InnerVolumeSpecName "kube-api-access-5vz2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:24.231023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.230982    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd" pod "fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" (UID: "fc273ca8-5ef5-4d77-b4aa-483a1bd9d910"). InnerVolumeSpecName "kube-api-access-gbvjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:24.328777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.328694    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vz2h\" (UniqueName: \"kubernetes.io/projected/c41e5112-952a-4a1d-841b-be3d0900b6dd-kube-api-access-5vz2h\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.328777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.328722    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41e5112-952a-4a1d-841b-be3d0900b6dd-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.328777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.328731    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.328777 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.328740    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbvjd\" (UniqueName: \"kubernetes.io/projected/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910-kube-api-access-gbvjd\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:24.332642 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.332600    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:24.664003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.663789    4577 generic.go:356] "Generic (PLEG): container finished" podID="fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" containerID="458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37" exitCode=0
Oct 30 01:55:24.664003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.663897    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h" event={"ID":"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910","Type":"ContainerDied","Data":"458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37"}
Oct 30 01:55:24.664003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.663971    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h" event={"ID":"fc273ca8-5ef5-4d77-b4aa-483a1bd9d910","Type":"ContainerDied","Data":"e1d9cd11232483751cb3e2b3f49de7f2962b25ca4cd03843c19f1d2d762ba0f6"}
Oct 30 01:55:24.664003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.663919    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"
Oct 30 01:55:24.664709 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.663995    4577 scope.go:122] "RemoveContainer" containerID="458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37"
Oct 30 01:55:24.666108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.666081    4577 generic.go:356] "Generic (PLEG): container finished" podID="c41e5112-952a-4a1d-841b-be3d0900b6dd" containerID="0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b" exitCode=0
Oct 30 01:55:24.666268 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.666246    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs" event={"ID":"c41e5112-952a-4a1d-841b-be3d0900b6dd","Type":"ContainerDied","Data":"0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b"}
Oct 30 01:55:24.666348 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.666287    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs" event={"ID":"c41e5112-952a-4a1d-841b-be3d0900b6dd","Type":"ContainerDied","Data":"51efdfcabe379073453d155e4a56a8dbc34adefc7c107f0593d63433fd45a5a6"}
Oct 30 01:55:24.666410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.666349    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"
Oct 30 01:55:24.671813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.671126    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"
Oct 30 01:55:24.671813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.671171    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg" event={"ID":"f852d0d3-9901-48da-af1f-f292a7731a96","Type":"ContainerDied","Data":"ff619860b8cc0f7790646f630c28aef515416040f2d067b7c112c78133976c02"}
Oct 30 01:55:24.678525 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.678502    4577 scope.go:122] "RemoveContainer" containerID="458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37"
Oct 30 01:55:24.678915 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:24.678893    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37\": container with ID starting with 458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37 not found: ID does not exist" containerID="458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37"
Oct 30 01:55:24.679001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.678921    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37"} err="failed to get container status \"458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37\": rpc error: code = NotFound desc = could not find container \"458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37\": container with ID starting with 458b91fdd63fd467ff6889c134fb94b8e82cecc14cddadb2f3c12c6c119fed37 not found: ID does not exist"
Oct 30 01:55:24.679001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.678939    4577 scope.go:122] "RemoveContainer" containerID="0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b"
Oct 30 01:55:24.691704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.691682    4577 scope.go:122] "RemoveContainer" containerID="0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b"
Oct 30 01:55:24.692047 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:24.692022    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b\": container with ID starting with 0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b not found: ID does not exist" containerID="0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b"
Oct 30 01:55:24.692139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.692056    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b"} err="failed to get container status \"0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b\": rpc error: code = NotFound desc = could not find container \"0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b\": container with ID starting with 0adddfbb1eb9837006e47c6ad2b602f0a28ebe1ee44a1a0efcf7ab0d43d04a7b not found: ID does not exist"
Oct 30 01:55:24.692139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.692079    4577 scope.go:122] "RemoveContainer" containerID="ed211d098f722c2f4de17af9af10a15441c034c9c29bd375c39553a6c7620844"
Oct 30 01:55:24.710748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.710723    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:24.725409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.725386    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"]
Oct 30 01:55:24.729160 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.729139    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-c666c2-7cd5c5d545-7sjtg"]
Oct 30 01:55:24.767812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.767363    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"]
Oct 30 01:55:24.772384 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.772354    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-72ed13-698f4c4654-mdk6h"]
Oct 30 01:55:24.824925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.824896    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"]
Oct 30 01:55:24.829492 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:24.829463    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-72ed13-5587dfd647-hkxfs"]
Oct 30 01:55:25.612136 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:25.612107    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:25.727046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:25.727011    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:26.110980 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.110942    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c41e5112-952a-4a1d-841b-be3d0900b6dd" path="/var/lib/kubelet/pods/c41e5112-952a-4a1d-841b-be3d0900b6dd/volumes"
Oct 30 01:55:26.111389 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.111374    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f852d0d3-9901-48da-af1f-f292a7731a96" path="/var/lib/kubelet/pods/f852d0d3-9901-48da-af1f-f292a7731a96/volumes"
Oct 30 01:55:26.111738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.111728    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" path="/var/lib/kubelet/pods/fc273ca8-5ef5-4d77-b4aa-483a1bd9d910/volumes"
Oct 30 01:55:26.113689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.113651    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:26.389529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.389448    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" containerID="cri-o://f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678" gracePeriod=45
Oct 30 01:55:26.522903 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.522871    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.66:8000/ready\": dial tcp 10.134.0.66:8000: connect: connection refused"
Oct 30 01:55:26.656746 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.656683    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 01:55:26.693748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.693717    4577 generic.go:356] "Generic (PLEG): container finished" podID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerID="f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678" exitCode=0
Oct 30 01:55:26.693937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.693774    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"
Oct 30 01:55:26.693937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.693806    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" event={"ID":"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5","Type":"ContainerDied","Data":"f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678"}
Oct 30 01:55:26.693937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.693875    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln" event={"ID":"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5","Type":"ContainerDied","Data":"b93a2c59a9f90302cf85930616caebf794a35d30290ce527ad7291694f297334"}
Oct 30 01:55:26.693937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.693890    4577 scope.go:122] "RemoveContainer" containerID="f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678"
Oct 30 01:55:26.704639 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.704621    4577 scope.go:122] "RemoveContainer" containerID="f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678"
Oct 30 01:55:26.705459 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:26.705008    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678\": container with ID starting with f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678 not found: ID does not exist" containerID="f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678"
Oct 30 01:55:26.705459 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.705050    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678"} err="failed to get container status \"f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678\": rpc error: code = NotFound desc = could not find container \"f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678\": container with ID starting with f21ea940ec94ea52c7bd85bac9dfb603d3dd56f248d80ca1eee7a9cda6e7b678 not found: ID does not exist"
Oct 30 01:55:26.760940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.760905    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs\" (UniqueName: \"kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs\") pod \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") "
Oct 30 01:55:26.761443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.760991    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph\" (UniqueName: \"kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph\") pod \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") "
Oct 30 01:55:26.761443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.761047    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir\") pod \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") "
Oct 30 01:55:26.761443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.761133    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm\") pod \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") "
Oct 30 01:55:26.761443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.761192    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home\") pod \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") "
Oct 30 01:55:26.761443 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.761225    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache\") pod \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\" (UID: \"7a6c86bb-9ba7-4602-abfb-df7fc8286ba5\") "
Oct 30 01:55:26.762188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.762155    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache" pod "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" (UID: "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:26.762331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.762183    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home" pod "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" (UID: "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:26.762331 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.762278    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir" pod "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" (UID: "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:26.765436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.765408    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm" pod "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" (UID: "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:26.765601 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.765569    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph" pod "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" (UID: "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5"). InnerVolumeSpecName "kube-api-access-9z5ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:26.768970 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.768943    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs" pod "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" (UID: "7a6c86bb-9ba7-4602-abfb-df7fc8286ba5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:26.862436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.862402    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:26.862436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.862429    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:26.862436 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.862439    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:26.862674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.862447    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z5ph\" (UniqueName: \"kubernetes.io/projected/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-kube-api-access-9z5ph\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:26.862674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.862474    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:26.862674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:26.862482    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:27.014323 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:27.014289    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:55:27.042681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:27.042646    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"]
Oct 30 01:55:27.048968 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:27.048943    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-664c2b/premium-w1-664c2b-kserve-7bdddd88c4-5xnln"]
Oct 30 01:55:27.445149 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:27.445060    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:27.857953 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:27.857922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:28.111602 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:28.111524    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" path="/var/lib/kubelet/pods/7a6c86bb-9ba7-4602-abfb-df7fc8286ba5/volumes"
Oct 30 01:55:28.410736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:28.410623    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:55:28.411474 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:28.411271    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:55:29.496183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496143    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"]
Oct 30 01:55:29.496876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496835    4577 cpu_manager.go:414] "containerMap: removing container" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main"
Oct 30 01:55:29.496876 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496876    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496891    4577 cpu_manager.go:414] "containerMap: removing container" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerName="payload-processing"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496896    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerName="payload-processing"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496911    4577 cpu_manager.go:414] "containerMap: removing container" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496917    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496923    4577 cpu_manager.go:414] "containerMap: removing container" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496928    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496935    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5aa19f12-8a54-4aad-9e86-b3344edac6db" containerName="maas-api"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496939    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5aa19f12-8a54-4aad-9e86-b3344edac6db" containerName="maas-api"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496949    4577 cpu_manager.go:414] "containerMap: removing container" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496953    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496960    4577 cpu_manager.go:414] "containerMap: removing container" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496965    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496976    4577 cpu_manager.go:414] "containerMap: removing container" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496980    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496986    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496990    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.496996    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497004    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497016    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f8c1a9a9-9b24-446b-b543-99e293cece15" containerName="main"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497023    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8c1a9a9-9b24-446b-b543-99e293cece15" containerName="main"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497033    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497038    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497045    4577 cpu_manager.go:414] "containerMap: removing container" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:55:29.497040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497049    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cf87902-7cb2-4647-aca3-33b9b7e65b6a" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497055    4577 cpu_manager.go:414] "containerMap: removing container" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497059    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497067    4577 cpu_manager.go:414] "containerMap: removing container" podUID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497071    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497080    4577 cpu_manager.go:414] "containerMap: removing container" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497084    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497090    4577 cpu_manager.go:414] "containerMap: removing container" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497095    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48323c68-17e2-4c68-9169-f7a61b27411f" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497102    4577 cpu_manager.go:414] "containerMap: removing container" podUID="38283fe4-8bfc-442b-b730-87b614feed0c" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497105    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38283fe4-8bfc-442b-b730-87b614feed0c" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497111    4577 cpu_manager.go:414] "containerMap: removing container" podUID="54abb8e7-cc15-4828-970b-3b0493136625" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497115    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54abb8e7-cc15-4828-970b-3b0493136625" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497123    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fb114e8e-a391-43ca-aaae-bf52d6518b37" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497128    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb114e8e-a391-43ca-aaae-bf52d6518b37" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497134    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f852d0d3-9901-48da-af1f-f292a7731a96" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497138    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f852d0d3-9901-48da-af1f-f292a7731a96" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497145    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497149    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497156    4577 cpu_manager.go:414] "containerMap: removing container" podUID="142ed92b-5389-4bff-96ba-683474be650f" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497160    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="142ed92b-5389-4bff-96ba-683474be650f" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497166    4577 cpu_manager.go:414] "containerMap: removing container" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497170    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497177    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497181    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497187    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497191    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497198    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497204    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497216    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497224    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497235    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497242    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497250    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497259    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497268    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497275    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497285    4577 cpu_manager.go:414] "containerMap: removing container" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497292    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497301    4577 cpu_manager.go:414] "containerMap: removing container" podUID="c41e5112-952a-4a1d-841b-be3d0900b6dd" containerName="payload-pre-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497307    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c41e5112-952a-4a1d-841b-be3d0900b6dd" containerName="payload-pre-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497333    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497340    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497348    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" containerName="payload-pre-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497355    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" containerName="payload-pre-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497368    4577 cpu_manager.go:414] "containerMap: removing container" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497375    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497383    4577 cpu_manager.go:414] "containerMap: removing container" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497389    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dabe84af-0e14-4545-a2a7-1c13a0814000" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497399    4577 cpu_manager.go:414] "containerMap: removing container" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497405    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497416    4577 cpu_manager.go:414] "containerMap: removing container" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497423    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497554    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497569    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497579    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c41e5112-952a-4a1d-841b-be3d0900b6dd" containerName="payload-pre-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497591    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a6c86bb-9ba7-4602-abfb-df7fc8286ba5" containerName="main"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497600    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a423130-f279-40f4-86fb-6120109a2a0b" containerName="payload-pre-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497613    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497620    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497630    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6050dd5-3567-411b-93cb-fe49d4fd7177" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497639    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf82a9c9-e781-431f-a03b-b0792078523c" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497648    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497656    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="44192915-065f-4bca-820e-e901202dfb53" containerName="main"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497666    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="421abe2b-d2f3-4455-81c9-1ac65acedd8f" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497679    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497690    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="88de8ed2-d09a-4715-85f4-35d8a9a51755" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497698    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497705    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5aa19f12-8a54-4aad-9e86-b3344edac6db" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497716    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497723    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497735    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45dc766-3e15-4a91-a753-de02ff246993" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497745    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c3d5d4d-b61b-41b6-8ce6-1b1eae82e29c" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497753    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="1baf66d7-a06c-41b0-af3a-47d379b7ae30" containerName="cleanup"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497759    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497768    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="da1bb547-5123-41b1-8a31-86b0b33b4837" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497775    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7b4a993-158d-453e-8818-aad628b254e9" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497782    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8c1a9a9-9b24-446b-b543-99e293cece15" containerName="main"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497792    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="55ceaeb2-4b48-44d1-8a1b-b4ca43008a53" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497802    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="38c3a078-0da3-44e6-91b9-32604ce48f3d" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497809    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="38283fe4-8bfc-442b-b730-87b614feed0c" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497818    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="54abb8e7-cc15-4828-970b-3b0493136625" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497829    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb114e8e-a391-43ca-aaae-bf52d6518b37" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497838    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f852d0d3-9901-48da-af1f-f292a7731a96" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497867    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="142ed92b-5389-4bff-96ba-683474be650f" containerName="maas-api"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497876    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497885    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc273ca8-5ef5-4d77-b4aa-483a1bd9d910" containerName="payload-processing"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497895    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497903    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497911    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2e195d5-f3d3-4e96-a1bc-6c898c909c63" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497919    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="2687b657-b370-4dc5-bf86-516d3111160f" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497927    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497933    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b1183c-5416-461c-9375-7e377bb762a1" containerName="istio-proxy"
Oct 30 01:55:29.498040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.497944    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9998a2ee-6eba-4438-8713-af86376d2dc2" containerName="istio-proxy"
Oct 30 01:55:29.503357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.503336    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.523776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.523747    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"]
Oct 30 01:55:29.588753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.588722    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 01:55:29.588991 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.588914    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:55:29.597730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.597704    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xwwk\" (UniqueName: \"kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.597928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.597910    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.597988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.597948    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.699518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.699481    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.699713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.699540    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.699713 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.699647    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xwwk\" (UniqueName: \"kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.700377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.700344    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.702234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.702214    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.725074 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.725039    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xwwk\" (UniqueName: \"kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk\") pod \"maas-api-revoke-sub-a8e3c6c5-s9jqt\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.736558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.736530    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:29.823650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.823560    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:29.850858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:29.850799    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:30.192245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:30.192209    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"]
Oct 30 01:55:30.192494 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:55:30.192466    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7f7273_efd5_40f4_830d_a8afe9367c34.slice/crio-d056d2ade08a9a6c90e4ed47e5c810328861e1d042141706ceced4ffb7da5625 WatchSource:0}: Error finding container d056d2ade08a9a6c90e4ed47e5c810328861e1d042141706ceced4ffb7da5625: Status 404 returned error can't find the container with id d056d2ade08a9a6c90e4ed47e5c810328861e1d042141706ceced4ffb7da5625
Oct 30 01:55:30.733829 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:30.733786    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt" event={"ID":"aa7f7273-efd5-40f4-830d-a8afe9367c34","Type":"ContainerStarted","Data":"80340a387606179015a4dde65811e1bfdbdce00ed15afbc64bfc07444a179e0c"}
Oct 30 01:55:30.734364 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:30.733901    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt" event={"ID":"aa7f7273-efd5-40f4-830d-a8afe9367c34","Type":"ContainerStarted","Data":"d056d2ade08a9a6c90e4ed47e5c810328861e1d042141706ceced4ffb7da5625"}
Oct 30 01:55:30.779956 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:30.779894    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt" podStartSLOduration=1.77987969 podStartE2EDuration="1.77987969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:55:30.777643805 +0000 UTC m=+4438.806954039" watchObservedRunningTime="2026-10-30 01:55:30.77987969 +0000 UTC m=+4438.809189917"
Oct 30 01:55:31.626541 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:31.626508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:31.745633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:31.745594    4577 generic.go:356] "Generic (PLEG): container finished" podID="aa7f7273-efd5-40f4-830d-a8afe9367c34" containerID="80340a387606179015a4dde65811e1bfdbdce00ed15afbc64bfc07444a179e0c" exitCode=0
Oct 30 01:55:31.746376 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:31.746230    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt" event={"ID":"aa7f7273-efd5-40f4-830d-a8afe9367c34","Type":"ContainerDied","Data":"80340a387606179015a4dde65811e1bfdbdce00ed15afbc64bfc07444a179e0c"}
Oct 30 01:55:31.811332 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:31.811294    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:32.447257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:32.447220    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:32.647375 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:32.647268    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:32.771910 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:32.771878    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"]
Oct 30 01:55:32.772370 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:32.772137    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" containerID="cri-o://266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929" gracePeriod=30
Oct 30 01:55:32.811688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:32.811647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:32.987902 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:32.984267    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:33.056425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.056389    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca\") pod \"aa7f7273-efd5-40f4-830d-a8afe9367c34\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") "
Oct 30 01:55:33.056640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.056583    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca\") pod \"aa7f7273-efd5-40f4-830d-a8afe9367c34\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") "
Oct 30 01:55:33.056712 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.056658    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk\" (UniqueName: \"kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk\") pod \"aa7f7273-efd5-40f4-830d-a8afe9367c34\" (UID: \"aa7f7273-efd5-40f4-830d-a8afe9367c34\") "
Oct 30 01:55:33.057271 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.057130    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca" pod "aa7f7273-efd5-40f4-830d-a8afe9367c34" (UID: "aa7f7273-efd5-40f4-830d-a8afe9367c34"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:33.060513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.060443    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk" pod "aa7f7273-efd5-40f4-830d-a8afe9367c34" (UID: "aa7f7273-efd5-40f4-830d-a8afe9367c34"). InnerVolumeSpecName "kube-api-access-2xwwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:33.061681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.061651    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca" pod "aa7f7273-efd5-40f4-830d-a8afe9367c34" (UID: "aa7f7273-efd5-40f4-830d-a8afe9367c34"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:33.158060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.158028    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:33.158060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.158057    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aa7f7273-efd5-40f4-830d-a8afe9367c34-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:33.158060 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.158068    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xwwk\" (UniqueName: \"kubernetes.io/projected/aa7f7273-efd5-40f4-830d-a8afe9367c34-kube-api-access-2xwwk\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:33.265689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.265656    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:33.623090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.623004    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:33.641385 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.641351    4577 patch_prober.go:29] interesting pod/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:55:33.641552 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.641412    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:55:33.764756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.764726    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt"
Oct 30 01:55:33.764756 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.764754    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8e3c6c5-s9jqt" event={"ID":"aa7f7273-efd5-40f4-830d-a8afe9367c34","Type":"ContainerDied","Data":"d056d2ade08a9a6c90e4ed47e5c810328861e1d042141706ceced4ffb7da5625"}
Oct 30 01:55:33.765085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:33.764805    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d056d2ade08a9a6c90e4ed47e5c810328861e1d042141706ceced4ffb7da5625"
Oct 30 01:55:35.520176 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:35.520144    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:37.461001 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:37.460962    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:38.069019 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.068635    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:55:38.227349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227310    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227361    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227400    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227535 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227466    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227721 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227595    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227731    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227764    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227801    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.227926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227808    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:38.227926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227825    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:38.227926 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227881    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:38.228163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.227973    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo\") pod \"fe335ad9-3734-4a58-a537-f9976621b4c6\" (UID: \"fe335ad9-3734-4a58-a537-f9976621b4c6\") "
Oct 30 01:55:38.228163 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228105    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:38.228269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228211    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:38.228574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228550    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.228574 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228574    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.228778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228587    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.228778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228599    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.228778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.228611    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe335ad9-3734-4a58-a537-f9976621b4c6-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.230473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.230451    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:55:38.230572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.230469    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:38.230572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.230497    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "kube-api-access-dxv6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:38.230698 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.230680    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token" pod "fe335ad9-3734-4a58-a537-f9976621b4c6" (UID: "fe335ad9-3734-4a58-a537-f9976621b4c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:38.329878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.329775    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.329878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.329820    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.329878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.329835    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxv6w\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-kube-api-access-dxv6w\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.329878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.329865    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe335ad9-3734-4a58-a537-f9976621b4c6-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:38.354629 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.354592    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"]
Oct 30 01:55:38.405784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.405742    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:55:38.406084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.405895    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:55:38.444769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.444743    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:38.802351 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.802313    4577 generic.go:356] "Generic (PLEG): container finished" podID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerID="266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929" exitCode=0
Oct 30 01:55:38.803000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.802370    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" event={"ID":"fe335ad9-3734-4a58-a537-f9976621b4c6","Type":"ContainerDied","Data":"266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929"}
Oct 30 01:55:38.803000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.802377    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"
Oct 30 01:55:38.803000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.802414    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb" event={"ID":"fe335ad9-3734-4a58-a537-f9976621b4c6","Type":"ContainerDied","Data":"47544d75b2563c724167d9ece6870d3d8f1c5b476221a5bb276d0d34862aefb4"}
Oct 30 01:55:38.803000 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.802436    4577 scope.go:122] "RemoveContainer" containerID="266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929"
Oct 30 01:55:38.814695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.814678    4577 scope.go:122] "RemoveContainer" containerID="96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"
Oct 30 01:55:38.824168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.824152    4577 scope.go:122] "RemoveContainer" containerID="266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929"
Oct 30 01:55:38.824429 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:38.824409    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929\": container with ID starting with 266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929 not found: ID does not exist" containerID="266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929"
Oct 30 01:55:38.824514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.824441    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929"} err="failed to get container status \"266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929\": rpc error: code = NotFound desc = could not find container \"266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929\": container with ID starting with 266678aba815238067e26e9691e9a60cac6b880925644f5f835261e4e38a8929 not found: ID does not exist"
Oct 30 01:55:38.824514 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.824467    4577 scope.go:122] "RemoveContainer" containerID="96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"
Oct 30 01:55:38.824703 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:38.824686    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816\": container with ID starting with 96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816 not found: ID does not exist" containerID="96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"
Oct 30 01:55:38.824747 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.824712    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816"} err="failed to get container status \"96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816\": rpc error: code = NotFound desc = could not find container \"96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816\": container with ID starting with 96ffd6b353e45cff156b51a026236bac2781d82ebc5dff9924ade6a69ca8e816 not found: ID does not exist"
Oct 30 01:55:38.863086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.863052    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"]
Oct 30 01:55:38.876710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:38.876674    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-c666c2-openshift-default-6747b8f7d5-k2gkb"]
Oct 30 01:55:39.350570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:39.350532    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:40.111624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:40.111585    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" path="/var/lib/kubelet/pods/fe335ad9-3734-4a58-a537-f9976621b4c6/volumes"
Oct 30 01:55:41.211391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:41.211353    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:42.376430 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.376388    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 01:55:42.397958 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.397930    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 01:55:42.486527 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.486492    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume\") pod \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") "
Oct 30 01:55:42.486729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.486659    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl\" (UniqueName: \"kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl\") pod \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\" (UID: \"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8\") "
Oct 30 01:55:42.487011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.486983    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume" pod "312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" (UID: "312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:42.489278 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.489252    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl" pod "312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" (UID: "312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8"). InnerVolumeSpecName "kube-api-access-5zvwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:42.588596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.588478    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf\" (UniqueName: \"kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf\") pod \"8083a038-6fca-44e5-bfc8-45af539c67dd\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") "
Oct 30 01:55:42.588596 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.588541    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume\") pod \"8083a038-6fca-44e5-bfc8-45af539c67dd\" (UID: \"8083a038-6fca-44e5-bfc8-45af539c67dd\") "
Oct 30 01:55:42.589088 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.589062    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:42.589216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.589094    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zvwl\" (UniqueName: \"kubernetes.io/projected/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8-kube-api-access-5zvwl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:42.589412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.589386    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume" pod "8083a038-6fca-44e5-bfc8-45af539c67dd" (UID: "8083a038-6fca-44e5-bfc8-45af539c67dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:42.593379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.592340    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf" pod "8083a038-6fca-44e5-bfc8-45af539c67dd" (UID: "8083a038-6fca-44e5-bfc8-45af539c67dd"). InnerVolumeSpecName "kube-api-access-s2kcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:42.613295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.613239    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:42.690090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.690059    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2kcf\" (UniqueName: \"kubernetes.io/projected/8083a038-6fca-44e5-bfc8-45af539c67dd-kube-api-access-s2kcf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:42.690090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.690089    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8083a038-6fca-44e5-bfc8-45af539c67dd-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:42.726674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.726627    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:42.833637 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.833602    4577 generic.go:356] "Generic (PLEG): container finished" podID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerID="375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314" exitCode=0
Oct 30 01:55:42.833805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.833663    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"
Oct 30 01:55:42.833805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.833687    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" event={"ID":"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8","Type":"ContainerDied","Data":"375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314"}
Oct 30 01:55:42.833805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.833753    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh" event={"ID":"312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8","Type":"ContainerDied","Data":"db02bea7f42c268b51bd678879554bc733e32236e24a67c43abbe42931219010"}
Oct 30 01:55:42.833805 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.833786    4577 scope.go:122] "RemoveContainer" containerID="375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314"
Oct 30 01:55:42.835378 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.835345    4577 generic.go:356] "Generic (PLEG): container finished" podID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerID="38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c" exitCode=0
Oct 30 01:55:42.835491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.835386    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" event={"ID":"8083a038-6fca-44e5-bfc8-45af539c67dd","Type":"ContainerDied","Data":"38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c"}
Oct 30 01:55:42.835491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.835408    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"
Oct 30 01:55:42.835491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.835436    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq" event={"ID":"8083a038-6fca-44e5-bfc8-45af539c67dd","Type":"ContainerDied","Data":"8f2463e1b4da9f40b65719a03d5bdc9ddf780f78d452633f81e6b3ac9d45e3e0"}
Oct 30 01:55:42.850262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.845365    4577 scope.go:122] "RemoveContainer" containerID="375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314"
Oct 30 01:55:42.850262 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:42.845711    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314\": container with ID starting with 375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314 not found: ID does not exist" containerID="375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314"
Oct 30 01:55:42.850262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.845747    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314"} err="failed to get container status \"375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314\": rpc error: code = NotFound desc = could not find container \"375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314\": container with ID starting with 375471fd613340746fc165eeb12601467428a084a50be6d8e9c53015ba26f314 not found: ID does not exist"
Oct 30 01:55:42.850262 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.845789    4577 scope.go:122] "RemoveContainer" containerID="38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c"
Oct 30 01:55:42.857243 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.857225    4577 scope.go:122] "RemoveContainer" containerID="38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c"
Oct 30 01:55:42.857549 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:42.857529    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c\": container with ID starting with 38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c not found: ID does not exist" containerID="38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c"
Oct 30 01:55:42.857624 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.857558    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c"} err="failed to get container status \"38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c\": rpc error: code = NotFound desc = could not find container \"38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c\": container with ID starting with 38a74d010dd0310fd7e71890b712c8e04c8bd07330086733e95b8b8487af264c not found: ID does not exist"
Oct 30 01:55:42.891230 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.891198    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"]
Oct 30 01:55:42.893930 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.893911    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-664c2b-7676856b8c-jlwvq"]
Oct 30 01:55:42.932085 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.932050    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"]
Oct 30 01:55:42.934723 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:42.934698    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-664c2b-575dbd9b58-4pghh"]
Oct 30 01:55:43.054267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:43.054228    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:43.177191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:43.177103    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:43.289023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:43.288992    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:43.716023 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:43.715989    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:43.842111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:43.842066    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:43.952871 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:43.952825    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:44.110584 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:44.110557    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" path="/var/lib/kubelet/pods/312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8/volumes"
Oct 30 01:55:44.110983 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:44.110970    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" path="/var/lib/kubelet/pods/8083a038-6fca-44e5-bfc8-45af539c67dd/volumes"
Oct 30 01:55:44.283304 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:44.283266    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:44.410677 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:44.410445    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:45.776718 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:45.776683    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:47.396618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.396572    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"]
Oct 30 01:55:47.397187 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.396822    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m" podUID="515d429f-3299-42c3-96e1-4cbe67ae6cbb" containerName="maas-api" containerID="cri-o://c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9" gracePeriod=30
Oct 30 01:55:47.450491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.450443    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:47.706315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.706291    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:55:47.782021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.781980    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls\") pod \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") "
Oct 30 01:55:47.782232 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.782205    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca\") pod \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") "
Oct 30 01:55:47.782322 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.782304    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs\") pod \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") "
Oct 30 01:55:47.782393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.782341    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7\" (UniqueName: \"kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7\") pod \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\" (UID: \"515d429f-3299-42c3-96e1-4cbe67ae6cbb\") "
Oct 30 01:55:47.782618 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.782590    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca" pod "515d429f-3299-42c3-96e1-4cbe67ae6cbb" (UID: "515d429f-3299-42c3-96e1-4cbe67ae6cbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:55:47.782923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.782894    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/515d429f-3299-42c3-96e1-4cbe67ae6cbb-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:47.784464 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.784434    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls" pod "515d429f-3299-42c3-96e1-4cbe67ae6cbb" (UID: "515d429f-3299-42c3-96e1-4cbe67ae6cbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:47.784663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.784638    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs" pod "515d429f-3299-42c3-96e1-4cbe67ae6cbb" (UID: "515d429f-3299-42c3-96e1-4cbe67ae6cbb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:47.784785 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.784767    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7" pod "515d429f-3299-42c3-96e1-4cbe67ae6cbb" (UID: "515d429f-3299-42c3-96e1-4cbe67ae6cbb"). InnerVolumeSpecName "kube-api-access-8hxz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:47.895894 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.891948    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"
Oct 30 01:55:47.904311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.901272    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m" event={"ID":"515d429f-3299-42c3-96e1-4cbe67ae6cbb","Type":"ContainerDied","Data":"c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9"}
Oct 30 01:55:47.904311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.901390    4577 scope.go:122] "RemoveContainer" containerID="c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9"
Oct 30 01:55:47.904311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.876907    4577 generic.go:356] "Generic (PLEG): container finished" podID="515d429f-3299-42c3-96e1-4cbe67ae6cbb" containerID="c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9" exitCode=0
Oct 30 01:55:47.904311 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.901656    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m" event={"ID":"515d429f-3299-42c3-96e1-4cbe67ae6cbb","Type":"ContainerDied","Data":"3f0e410d373cdade0ba1fd878622da8b56f615ef67d4360248843740ebae9658"}
Oct 30 01:55:47.907264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.905594    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:47.907264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.905622    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hxz7\" (UniqueName: \"kubernetes.io/projected/515d429f-3299-42c3-96e1-4cbe67ae6cbb-kube-api-access-8hxz7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:47.907264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.905637    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/515d429f-3299-42c3-96e1-4cbe67ae6cbb-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:47.956529 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.956311    4577 scope.go:122] "RemoveContainer" containerID="c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9"
Oct 30 01:55:47.957127 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:47.957040    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9\": container with ID starting with c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9 not found: ID does not exist" containerID="c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9"
Oct 30 01:55:47.957127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:47.957081    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9"} err="failed to get container status \"c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9\": rpc error: code = NotFound desc = could not find container \"c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9\": container with ID starting with c4d8ce0fa9b85835c331386f1775e3d73a84ac2ffa7dd88e2bdd5ea76a1008f9 not found: ID does not exist"
Oct 30 01:55:48.023197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.023153    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"]
Oct 30 01:55:48.026395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.026371    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-2f820e-779b454cd5-9584m"]
Oct 30 01:55:48.070379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.070260    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:48.112391 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.112363    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="515d429f-3299-42c3-96e1-4cbe67ae6cbb" path="/var/lib/kubelet/pods/515d429f-3299-42c3-96e1-4cbe67ae6cbb/volumes"
Oct 30 01:55:48.235465 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.235415    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:48.378341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.378279    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:48.668504 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:48.668394    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:50.322526 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:50.322494    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:50.449682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:50.449647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:51.988988 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:51.988954    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:52.304185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:52.304088    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:52.678689 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:52.678598    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:52.809011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:52.808922    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:53.024252 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.024218    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:53.201867 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.201447    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:53.320817 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.320039    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:53.377343 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.377289    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main" containerID="cri-o://2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc" gracePeriod=45
Oct 30 01:55:53.438495 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.438464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:53.587016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.586947    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:53.713870 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.706707    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:55:53.798349 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798322    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache\") pod \"fd122b33-6b60-4103-a003-a8d6cdf84f27\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") "
Oct 30 01:55:53.798577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798364    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home\") pod \"fd122b33-6b60-4103-a003-a8d6cdf84f27\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") "
Oct 30 01:55:53.798577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798401    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir\") pod \"fd122b33-6b60-4103-a003-a8d6cdf84f27\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") "
Oct 30 01:55:53.798577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798479    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r\" (UniqueName: \"kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r\") pod \"fd122b33-6b60-4103-a003-a8d6cdf84f27\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") "
Oct 30 01:55:53.798577 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798521    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs\") pod \"fd122b33-6b60-4103-a003-a8d6cdf84f27\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") "
Oct 30 01:55:53.800818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798651    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache" pod "fd122b33-6b60-4103-a003-a8d6cdf84f27" (UID: "fd122b33-6b60-4103-a003-a8d6cdf84f27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:53.800818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798684    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm\") pod \"fd122b33-6b60-4103-a003-a8d6cdf84f27\" (UID: \"fd122b33-6b60-4103-a003-a8d6cdf84f27\") "
Oct 30 01:55:53.800818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.798885    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir" pod "fd122b33-6b60-4103-a003-a8d6cdf84f27" (UID: "fd122b33-6b60-4103-a003-a8d6cdf84f27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:53.800818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.799187    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:53.800818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.799206    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:53.800818 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.799427    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home" pod "fd122b33-6b60-4103-a003-a8d6cdf84f27" (UID: "fd122b33-6b60-4103-a003-a8d6cdf84f27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:53.802542 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.802482    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs" pod "fd122b33-6b60-4103-a003-a8d6cdf84f27" (UID: "fd122b33-6b60-4103-a003-a8d6cdf84f27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:55:53.802681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.802649    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm" pod "fd122b33-6b60-4103-a003-a8d6cdf84f27" (UID: "fd122b33-6b60-4103-a003-a8d6cdf84f27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:55:53.809949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.809513    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r" pod "fd122b33-6b60-4103-a003-a8d6cdf84f27" (UID: "fd122b33-6b60-4103-a003-a8d6cdf84f27"). InnerVolumeSpecName "kube-api-access-hnd5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:55:53.900406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.900321    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:53.900406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.900364    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnd5r\" (UniqueName: \"kubernetes.io/projected/fd122b33-6b60-4103-a003-a8d6cdf84f27-kube-api-access-hnd5r\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:53.900406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.900379    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd122b33-6b60-4103-a003-a8d6cdf84f27-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:53.900406 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.900392    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd122b33-6b60-4103-a003-a8d6cdf84f27-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:55:53.938695 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.938663    4577 generic.go:356] "Generic (PLEG): container finished" podID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerID="2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc" exitCode=0
Oct 30 01:55:53.938929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.938706    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"
Oct 30 01:55:53.938929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.938748    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" event={"ID":"fd122b33-6b60-4103-a003-a8d6cdf84f27","Type":"ContainerDied","Data":"2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc"}
Oct 30 01:55:53.938929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.938794    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx" event={"ID":"fd122b33-6b60-4103-a003-a8d6cdf84f27","Type":"ContainerDied","Data":"42906189cc8bb60d13881ffdd4b61be0f9d2b9b3de9b680826e81aa36d0984f9"}
Oct 30 01:55:53.938929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.938815    4577 scope.go:122] "RemoveContainer" containerID="2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc"
Oct 30 01:55:53.951011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.950982    4577 scope.go:122] "RemoveContainer" containerID="2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc"
Oct 30 01:55:53.951404 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:53.951369    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc\": container with ID starting with 2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc not found: ID does not exist" containerID="2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc"
Oct 30 01:55:53.951543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.951414    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc"} err="failed to get container status \"2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc\": rpc error: code = NotFound desc = could not find container \"2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc\": container with ID starting with 2c5df6bc59011314ad7206532015e0b366e0c0034b725475a6200b82ae52c2bc not found: ID does not exist"
Oct 30 01:55:53.998044 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:53.998003    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"]
Oct 30 01:55:54.046872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:54.031129    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9bfe36/premium-w0-9bfe36-kserve-8db555955-lpfhx"]
Oct 30 01:55:54.111190 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:54.111151    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" path="/var/lib/kubelet/pods/fd122b33-6b60-4103-a003-a8d6cdf84f27/volumes"
Oct 30 01:55:54.222986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:54.222899    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:56.102556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:56.102511    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:56.229069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:56.229034    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:57.014148 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:55:57.014113    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:55:57.138116 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:57.138075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:57.269786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:57.269698    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:57.469573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:57.469541    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:57.959227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:57.959194    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:59.074108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:59.074070    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:55:59.916620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:55:59.916585    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:00.341131 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:00.341102    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:01.902472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:01.902440    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:03.605184 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:03.605151    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:05.388303 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:05.388267    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:06.856183 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:06.856150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:07.393722 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:07.393682    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:07.505070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:07.505035    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:07.625457 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:07.625418    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:08.017645 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:08.017613    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:08.600621 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:08.600589    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:08.718315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:08.718272    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:09.423216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:09.423179    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:11.312265 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:11.312235    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:13.074325 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:13.074270    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:14.679986 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:14.679943    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:14.797759 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:14.797724    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:14.910119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:14.910075    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:15.021150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:15.021120    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:16.338090 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:16.338059    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:16.794501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:16.794461    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:16.911752 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:16.911689    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:17.031011 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:17.030974    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:17.166594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:17.166508    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:17.627335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:17.627293    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:17.752027 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:17.751988    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:19.008797 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:19.008763    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:20.712643 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:20.712604    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:22.516657 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:22.516617    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:24.262834 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:24.262805    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:24.388764 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:24.388724    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:24.561688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:24.561603    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"]
Oct 30 01:56:24.561940 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:24.561893    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy" containerID="cri-o://06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c" gracePeriod=30
Oct 30 01:56:24.612014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:24.611982    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:24.883609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:24.883527    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:25.720402 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:25.720375    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:26.182344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:26.182301    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:27.015901 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:27.015861    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:56:27.563377 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:27.563346    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:27.994373 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:27.994334    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:28.520620 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:28.520583    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:28.643681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:28.643647    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:29.821238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.821065    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:29.850197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.850126    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:56:29.877825 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.877786    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.877890    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.877920    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878057 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.878009    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.878106    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.878167    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878244 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.878209    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.878253    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.878410 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.878286    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc\") pod \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\" (UID: \"172b3d82-5c75-4c05-8f65-1af8ca1a277d\") "
Oct 30 01:56:29.879520 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.879486    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:56:29.879685 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.879649    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:29.879951 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.879750    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:29.880150 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.880126    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:29.880573 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.880320    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:29.884016 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.883994    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:56:29.884277 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.884250    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:29.884720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.884699    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:56:29.884795 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.884727    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc" pod "172b3d82-5c75-4c05-8f65-1af8ca1a277d" (UID: "172b3d82-5c75-4c05-8f65-1af8ca1a277d"). InnerVolumeSpecName "kube-api-access-9r2qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:56:29.979627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979592    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979619    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979627 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979632    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979644    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r2qc\" (UniqueName: \"kubernetes.io/projected/172b3d82-5c75-4c05-8f65-1af8ca1a277d-kube-api-access-9r2qc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979660    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979671    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979683    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979693    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:29.979918 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:29.979704    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/172b3d82-5c75-4c05-8f65-1af8ca1a277d-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:30.211340 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.211309    4577 generic.go:356] "Generic (PLEG): container finished" podID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerID="06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c" exitCode=0
Oct 30 01:56:30.211521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.211366    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"
Oct 30 01:56:30.211521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.211397    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" event={"ID":"172b3d82-5c75-4c05-8f65-1af8ca1a277d","Type":"ContainerDied","Data":"06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c"}
Oct 30 01:56:30.211521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.211451    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q" event={"ID":"172b3d82-5c75-4c05-8f65-1af8ca1a277d","Type":"ContainerDied","Data":"228e060f6ba01a06a0a8d0779e874e0da1dd704868cfad51ab9a17641d71b1b1"}
Oct 30 01:56:30.211521 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.211474    4577 scope.go:122] "RemoveContainer" containerID="06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c"
Oct 30 01:56:30.221636 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.221619    4577 scope.go:122] "RemoveContainer" containerID="813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"
Oct 30 01:56:30.230836 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.230818    4577 scope.go:122] "RemoveContainer" containerID="06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c"
Oct 30 01:56:30.231169 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:30.231150    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c\": container with ID starting with 06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c not found: ID does not exist" containerID="06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c"
Oct 30 01:56:30.231224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.231179    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c"} err="failed to get container status \"06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c\": rpc error: code = NotFound desc = could not find container \"06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c\": container with ID starting with 06b5b9755e156f747a0be2025d975ea4271e5bcd9249f3e38bf3c9417f6b847c not found: ID does not exist"
Oct 30 01:56:30.231224 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.231196    4577 scope.go:122] "RemoveContainer" containerID="813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"
Oct 30 01:56:30.231434 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:30.231418    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7\": container with ID starting with 813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7 not found: ID does not exist" containerID="813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"
Oct 30 01:56:30.231473 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.231443    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7"} err="failed to get container status \"813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7\": rpc error: code = NotFound desc = could not find container \"813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7\": container with ID starting with 813fb8b05d45f99097997b0b499506897f8911c5ba2c19bd8941916715534aa7 not found: ID does not exist"
Oct 30 01:56:30.271803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.271773    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"]
Oct 30 01:56:30.285496 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.285462    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-1afb3e-openshift-default-7c4c64d649-xlx9q"]
Oct 30 01:56:30.852559 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.852524    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:30.966513 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:30.966471    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:31.601237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:31.601206    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:32.110357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:32.110319    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" path="/var/lib/kubelet/pods/172b3d82-5c75-4c05-8f65-1af8ca1a277d/volumes"
Oct 30 01:56:33.407038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:33.406997    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:35.209266 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:35.209233    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:35.553481 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:35.553451    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"]
Oct 30 01:56:35.585794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:35.585763    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"]
Oct 30 01:56:35.689729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:35.689699    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:35.800976 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:35.800940    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:37.005556 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:37.005529    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:38.725077 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:38.725036    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:40.175632 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.175600    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"]
Oct 30 01:56:40.176126 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.175837    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" podUID="5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" containerName="maas-api" containerID="cri-o://1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917" gracePeriod=30
Oct 30 01:56:40.241808 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.241779    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:40.464553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.464205    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:56:40.510395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.510366    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh\" (UniqueName: \"kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh\") pod \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") "
Oct 30 01:56:40.510616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.510463    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\") pod \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") "
Oct 30 01:56:40.510616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.510541    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca\") pod \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") "
Oct 30 01:56:40.510616 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.510571    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs\") pod \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\" (UID: \"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a\") "
Oct 30 01:56:40.511003 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.510966    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca" pod "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" (UID: "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:56:40.512766 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.512732    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh" pod "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" (UID: "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a"). InnerVolumeSpecName "kube-api-access-55vnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:56:40.513021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.512992    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls" pod "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" (UID: "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:56:40.513021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.513002    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs" pod "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" (UID: "5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:56:40.553729 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.553690    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing" containerID="cri-o://9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32" gracePeriod=25
Oct 30 01:56:40.586122 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.586061    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing" containerID="cri-o://e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7" gracePeriod=25
Oct 30 01:56:40.612192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.612151    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:40.612192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.612192    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:40.612192 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.612204    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:40.612442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.612216    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55vnh\" (UniqueName: \"kubernetes.io/projected/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a-kube-api-access-55vnh\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:40.792530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.792492    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:40.816161 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.816123    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:56:40.816368 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.816132    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:56:40.906128 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:40.906092    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:41.301917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.301884    4577 generic.go:356] "Generic (PLEG): container finished" podID="5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" containerID="1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917" exitCode=0
Oct 30 01:56:41.302361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.301937    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"
Oct 30 01:56:41.302361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.301940    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" event={"ID":"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a","Type":"ContainerDied","Data":"1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917"}
Oct 30 01:56:41.302361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.302073    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd" event={"ID":"5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a","Type":"ContainerDied","Data":"b2206d78c6fb1e50b7b8f78e6b119445a641716c795d24dd8417dacc75d5dcde"}
Oct 30 01:56:41.302361 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.302108    4577 scope.go:122] "RemoveContainer" containerID="1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917"
Oct 30 01:56:41.312169 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.312149    4577 scope.go:122] "RemoveContainer" containerID="1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917"
Oct 30 01:56:41.312451 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:41.312432    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917\": container with ID starting with 1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917 not found: ID does not exist" containerID="1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917"
Oct 30 01:56:41.312502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.312459    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917"} err="failed to get container status \"1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917\": rpc error: code = NotFound desc = could not find container \"1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917\": container with ID starting with 1e8a8d3621c0f26dfcabdcee2048277f5f74fd3c489efd6ac232ac962f845917 not found: ID does not exist"
Oct 30 01:56:41.359594 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.359571    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"]
Oct 30 01:56:41.373372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.373349    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-4107fc-7c45b477d5-f6fjd"]
Oct 30 01:56:41.407335 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:41.407311    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:42.110484 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:42.110452    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" path="/var/lib/kubelet/pods/5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a/volumes"
Oct 30 01:56:42.708935 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:42.708832    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:44.518021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:44.517985    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:46.314680 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:46.314649    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:48.116731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:48.116694    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:48.347075 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:48.347042    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:48.461671 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:48.461581    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:48.730138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:48.722603    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"]
Oct 30 01:56:48.730138 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:48.722985    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" containerID="cri-o://31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d" gracePeriod=30
Oct 30 01:56:48.793821 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:48.793786    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:49.992072 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:49.992039    4577 patch_prober.go:29] interesting pod/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:56:49.992491 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:49.992108    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:56:50.041963 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:50.041919    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:50.816272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:50.816231    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:56:50.816272 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:50.816254    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:56:51.808732 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:51.808693    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:53.605177 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:53.605117    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:53.989588 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:53.989560    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:56:54.084833 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.084798    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.084905    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.084937    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.084981    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085033 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085020    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085068    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085126    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085166    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085207    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh\") pod \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\" (UID: \"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb\") "
Oct 30 01:56:54.085257 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085237    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:54.085575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085341    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:54.085575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085378    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:56:54.085575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085419    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:54.085575 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085506    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:54.085778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085704    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.085778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085717    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.085778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085729    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.085778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085746    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.085778 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.085763    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.087315 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.087285    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:56:54.087533 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.087513    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:56:54.087929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.087905    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:56:54.087929 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.087914    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh" pod "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" (UID: "e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb"). InnerVolumeSpecName "kube-api-access-6ntvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:56:54.187140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.187062    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.187140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.187090    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.187140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.187102    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ntvh\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-kube-api-access-6ntvh\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.187140 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.187115    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:56:54.379704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.379664    4577 generic.go:356] "Generic (PLEG): container finished" podID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerID="31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d" exitCode=0
Oct 30 01:56:54.379941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.379718    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"
Oct 30 01:56:54.379941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.379742    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" event={"ID":"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb","Type":"ContainerDied","Data":"31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d"}
Oct 30 01:56:54.379941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.379806    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm" event={"ID":"e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb","Type":"ContainerDied","Data":"67a3b3f93dca600e83b7d758cf93ceb055bc66d18cbc94c195f0f8c0c688e25f"}
Oct 30 01:56:54.379941 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.379829    4577 scope.go:122] "RemoveContainer" containerID="31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d"
Oct 30 01:56:54.390173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.390148    4577 scope.go:122] "RemoveContainer" containerID="7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"
Oct 30 01:56:54.399186 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.399171    4577 scope.go:122] "RemoveContainer" containerID="31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d"
Oct 30 01:56:54.399455 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:54.399435    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d\": container with ID starting with 31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d not found: ID does not exist" containerID="31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d"
Oct 30 01:56:54.399516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.399465    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d"} err="failed to get container status \"31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d\": rpc error: code = NotFound desc = could not find container \"31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d\": container with ID starting with 31880d9af26e03c9a537bfdf7ca013025c0a3244cf5f65a597a2627db6d3f92d not found: ID does not exist"
Oct 30 01:56:54.399516 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.399482    4577 scope.go:122] "RemoveContainer" containerID="7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"
Oct 30 01:56:54.399739 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:54.399722    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3\": container with ID starting with 7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3 not found: ID does not exist" containerID="7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"
Oct 30 01:56:54.399784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.399747    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3"} err="failed to get container status \"7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3\": rpc error: code = NotFound desc = could not find container \"7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3\": container with ID starting with 7a097debdd15c614ad6b0b779f58713803c499bd524a1ba35d5a653ddaf9d6f3 not found: ID does not exist"
Oct 30 01:56:54.438207 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.438147    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"]
Oct 30 01:56:54.452178 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:54.452156    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-4107fc-openshift-default-64f47f9f5c-g2ngm"]
Oct 30 01:56:55.313715 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:55.313675    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:56.110904 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:56.110834    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" path="/var/lib/kubelet/pods/e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb/volumes"
Oct 30 01:56:57.014210 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:56:57.014169    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:56:57.019740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:57.019714    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:58.764503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:58.764435    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:59.826830 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:59.826784    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:56:59.937227 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:56:59.937168    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:00.523998 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.523956    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:00.842042 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.842010    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:57:00.876021 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.875996    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:57:00.959191 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.959158    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume\") pod \"853ec6ff-84b8-4179-b14a-7e50f91534a9\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") "
Oct 30 01:57:00.959412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.959233    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq\" (UniqueName: \"kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq\") pod \"853ec6ff-84b8-4179-b14a-7e50f91534a9\" (UID: \"853ec6ff-84b8-4179-b14a-7e50f91534a9\") "
Oct 30 01:57:00.959412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.959309    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume\") pod \"ec96526a-90f6-4096-a8e7-209f0740c279\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") "
Oct 30 01:57:00.959553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.959419    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl\" (UniqueName: \"kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl\") pod \"ec96526a-90f6-4096-a8e7-209f0740c279\" (UID: \"ec96526a-90f6-4096-a8e7-209f0740c279\") "
Oct 30 01:57:00.959704 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.959671    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume" pod "853ec6ff-84b8-4179-b14a-7e50f91534a9" (UID: "853ec6ff-84b8-4179-b14a-7e50f91534a9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:00.959784 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.959713    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume" pod "ec96526a-90f6-4096-a8e7-209f0740c279" (UID: "ec96526a-90f6-4096-a8e7-209f0740c279"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:00.961623 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.961599    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl" pod "ec96526a-90f6-4096-a8e7-209f0740c279" (UID: "ec96526a-90f6-4096-a8e7-209f0740c279"). InnerVolumeSpecName "kube-api-access-t9mdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:00.962121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:00.962092    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq" pod "853ec6ff-84b8-4179-b14a-7e50f91534a9" (UID: "853ec6ff-84b8-4179-b14a-7e50f91534a9"). InnerVolumeSpecName "kube-api-access-7t9lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:01.060793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.060707    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/853ec6ff-84b8-4179-b14a-7e50f91534a9-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:01.060793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.060738    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t9lq\" (UniqueName: \"kubernetes.io/projected/853ec6ff-84b8-4179-b14a-7e50f91534a9-kube-api-access-7t9lq\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:01.060793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.060747    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96526a-90f6-4096-a8e7-209f0740c279-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:01.060793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.060756    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9mdl\" (UniqueName: \"kubernetes.io/projected/ec96526a-90f6-4096-a8e7-209f0740c279-kube-api-access-t9mdl\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:01.422155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.422067    4577 generic.go:356] "Generic (PLEG): container finished" podID="ec96526a-90f6-4096-a8e7-209f0740c279" containerID="e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7" exitCode=0
Oct 30 01:57:01.422155 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.422124    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"
Oct 30 01:57:01.422393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.422160    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" event={"ID":"ec96526a-90f6-4096-a8e7-209f0740c279","Type":"ContainerDied","Data":"e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7"}
Oct 30 01:57:01.422393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.422224    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" event={"ID":"ec96526a-90f6-4096-a8e7-209f0740c279","Type":"ContainerDied","Data":"d20cb8412a323c480ee09e3111a3fdfeef2af72a6adcc14feecefbb47960fe6d"}
Oct 30 01:57:01.422393 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.422250    4577 scope.go:122] "RemoveContainer" containerID="e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7"
Oct 30 01:57:01.423585 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.423560    4577 generic.go:356] "Generic (PLEG): container finished" podID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerID="9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32" exitCode=0
Oct 30 01:57:01.423714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.423611    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"
Oct 30 01:57:01.423714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.423641    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" event={"ID":"853ec6ff-84b8-4179-b14a-7e50f91534a9","Type":"ContainerDied","Data":"9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32"}
Oct 30 01:57:01.423714 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.423693    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" event={"ID":"853ec6ff-84b8-4179-b14a-7e50f91534a9","Type":"ContainerDied","Data":"ff4243e23d2cca178b1a4ebea8c802d56ad711527a1a3c558fe20629e125193e"}
Oct 30 01:57:01.433772 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.433660    4577 scope.go:122] "RemoveContainer" containerID="e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7"
Oct 30 01:57:01.434156 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:01.434046    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7\": container with ID starting with e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7 not found: ID does not exist" containerID="e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7"
Oct 30 01:57:01.434156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.434084    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7"} err="failed to get container status \"e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7\": rpc error: code = NotFound desc = could not find container \"e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7\": container with ID starting with e052473698967c97a7669622262029c4c2c1acb287caf082783fa7b963be4aa7 not found: ID does not exist"
Oct 30 01:57:01.434156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.434109    4577 scope.go:122] "RemoveContainer" containerID="9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32"
Oct 30 01:57:01.444295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.444277    4577 scope.go:122] "RemoveContainer" containerID="9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32"
Oct 30 01:57:01.444570 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:01.444549    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32\": container with ID starting with 9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32 not found: ID does not exist" containerID="9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32"
Oct 30 01:57:01.444662 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.444576    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32"} err="failed to get container status \"9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32\": rpc error: code = NotFound desc = could not find container \"9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32\": container with ID starting with 9b985c6db7613315bd1e4715bacc7944054df09e3cc3e92bfca14af2910c2d32 not found: ID does not exist"
Oct 30 01:57:01.480471 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.480439    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"]
Oct 30 01:57:01.494103 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.494080    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb"]
Oct 30 01:57:01.531649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.531622    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"]
Oct 30 01:57:01.534663 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.534641    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4"]
Oct 30 01:57:01.815566 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.815527    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-4107fc-6c8fff4b4c-n78h4" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.221:50052\" within 1s: context deadline exceeded"
Oct 30 01:57:01.815925 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:57:01.815604    4577 logging.go:55] [core] [Channel #11880 SubChannel #11882]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.221:50052", ServerName: "10.134.0.221:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.221:50052: operation was canceled"
Oct 30 01:57:01.815925 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:01.815696    4577 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-4107fc-675db85bb6-ns5qb" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.220:50052\" within 1s: context deadline exceeded"
Oct 30 01:57:01.815925 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:57:01.815769    4577 logging.go:55] [core] [Channel #11881 SubChannel #11883]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.220:50052", ServerName: "10.134.0.220:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.220:50052: operation was canceled"
Oct 30 01:57:02.024898 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.024864    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"]
Oct 30 01:57:02.025412 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.025104    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr" podUID="9efeba0b-699f-4979-b51b-55ca71b66736" containerName="maas-api" containerID="cri-o://b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709" gracePeriod=30
Oct 30 01:57:02.109882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.109756    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:02.112008 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.111979    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" path="/var/lib/kubelet/pods/853ec6ff-84b8-4179-b14a-7e50f91534a9/volumes"
Oct 30 01:57:02.112463 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.112442    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" path="/var/lib/kubelet/pods/ec96526a-90f6-4096-a8e7-209f0740c279/volumes"
Oct 30 01:57:02.316197 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.316171    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:57:02.375045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.374951    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs\") pod \"9efeba0b-699f-4979-b51b-55ca71b66736\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") "
Oct 30 01:57:02.375045 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.375027    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca\") pod \"9efeba0b-699f-4979-b51b-55ca71b66736\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") "
Oct 30 01:57:02.375320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.375099    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv\" (UniqueName: \"kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv\") pod \"9efeba0b-699f-4979-b51b-55ca71b66736\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") "
Oct 30 01:57:02.375320 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.375274    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls\") pod \"9efeba0b-699f-4979-b51b-55ca71b66736\" (UID: \"9efeba0b-699f-4979-b51b-55ca71b66736\") "
Oct 30 01:57:02.375503 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.375472    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca" pod "9efeba0b-699f-4979-b51b-55ca71b66736" (UID: "9efeba0b-699f-4979-b51b-55ca71b66736"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:02.375748 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.375726    4577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efeba0b-699f-4979-b51b-55ca71b66736-openshift-service-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:02.377655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.377630    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs" pod "9efeba0b-699f-4979-b51b-55ca71b66736" (UID: "9efeba0b-699f-4979-b51b-55ca71b66736"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:57:02.378299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.378280    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls" pod "9efeba0b-699f-4979-b51b-55ca71b66736" (UID: "9efeba0b-699f-4979-b51b-55ca71b66736"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:57:02.378449 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.378431    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv" pod "9efeba0b-699f-4979-b51b-55ca71b66736" (UID: "9efeba0b-699f-4979-b51b-55ca71b66736"). InnerVolumeSpecName "kube-api-access-pgqfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:02.432537 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.432507    4577 generic.go:356] "Generic (PLEG): container finished" podID="9efeba0b-699f-4979-b51b-55ca71b66736" containerID="b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709" exitCode=0
Oct 30 01:57:02.432682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.432563    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"
Oct 30 01:57:02.432682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.432595    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr" event={"ID":"9efeba0b-699f-4979-b51b-55ca71b66736","Type":"ContainerDied","Data":"b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709"}
Oct 30 01:57:02.432682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.432640    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr" event={"ID":"9efeba0b-699f-4979-b51b-55ca71b66736","Type":"ContainerDied","Data":"e6c0b92d82ec32fb93db927830f7d203ca2df16231a3e39b65d5f6d73b73004c"}
Oct 30 01:57:02.432682 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.432662    4577 scope.go:122] "RemoveContainer" containerID="b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709"
Oct 30 01:57:02.442939 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.442923    4577 scope.go:122] "RemoveContainer" containerID="b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709"
Oct 30 01:57:02.443205 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:02.443179    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709\": container with ID starting with b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709 not found: ID does not exist" containerID="b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709"
Oct 30 01:57:02.443267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.443205    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709"} err="failed to get container status \"b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709\": rpc error: code = NotFound desc = could not find container \"b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709\": container with ID starting with b546a62163008edfb6f7dc18bbc2e0788161e97d128203f3ee3186ef416af709 not found: ID does not exist"
Oct 30 01:57:02.482420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.481808    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-maas-api-tls\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:02.482420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.481882    4577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efeba0b-699f-4979-b51b-55ca71b66736-metrics-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:02.482420 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.481899    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqfv\" (UniqueName: \"kubernetes.io/projected/9efeba0b-699f-4979-b51b-55ca71b66736-kube-api-access-pgqfv\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:02.500975 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.500896    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"]
Oct 30 01:57:02.508872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.505127    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-bc127f-7f7f45cf76-8l7lr"]
Oct 30 01:57:02.643395 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.643319    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:02.894646 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:02.894531    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:03.328018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:03.327984    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:04.110223 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:04.110193    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9efeba0b-699f-4979-b51b-55ca71b66736" path="/var/lib/kubelet/pods/9efeba0b-699f-4979-b51b-55ca71b66736/volumes"
Oct 30 01:57:04.706736 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:04.706689    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:06.411200 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:06.411166    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:08.110390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:08.110362    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:09.574792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:09.574755    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:09.691570 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:09.691540    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:09.894899 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:09.894769    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"]
Oct 30 01:57:09.895078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:09.895041    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy" containerID="cri-o://bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca" gracePeriod=30
Oct 30 01:57:09.916993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:09.916970    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:10.028811 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.028772    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:10.381040 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.381002    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:10.603649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.603613    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"]
Oct 30 01:57:10.604108 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604093    4577 cpu_manager.go:414] "containerMap: removing container" podUID="515d429f-3299-42c3-96e1-4cbe67ae6cbb" containerName="maas-api"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604109    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="515d429f-3299-42c3-96e1-4cbe67ae6cbb" containerName="maas-api"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604121    4577 cpu_manager.go:414] "containerMap: removing container" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604126    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa6ec37d-8703-4491-a295-91eb0e031794" containerName="istio-proxy"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604134    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604139    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604146    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy"
Oct 30 01:57:10.604154 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604151    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604158    4577 cpu_manager.go:414] "containerMap: removing container" podUID="5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" containerName="maas-api"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604163    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" containerName="maas-api"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604170    4577 cpu_manager.go:414] "containerMap: removing container" podUID="aa7f7273-efd5-40f4-830d-a8afe9367c34" containerName="revoke-keys"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604174    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa7f7273-efd5-40f4-830d-a8afe9367c34" containerName="revoke-keys"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604180    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604185    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604193    4577 cpu_manager.go:414] "containerMap: removing container" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerName="payload-pre-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604197    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerName="payload-pre-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604205    4577 cpu_manager.go:414] "containerMap: removing container" podUID="9efeba0b-699f-4979-b51b-55ca71b66736" containerName="maas-api"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604210    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9efeba0b-699f-4979-b51b-55ca71b66736" containerName="maas-api"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604218    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604222    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604236    4577 cpu_manager.go:414] "containerMap: removing container" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604241    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604248    4577 cpu_manager.go:414] "containerMap: removing container" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604252    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604260    4577 cpu_manager.go:414] "containerMap: removing container" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604263    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604269    4577 cpu_manager.go:414] "containerMap: removing container" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604273    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92ddec96-952c-4a03-ae26-09aa2368fdfb" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604279    4577 cpu_manager.go:414] "containerMap: removing container" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604283    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604288    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604293    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604300    4577 cpu_manager.go:414] "containerMap: removing container" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604304    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62f086f0-79fa-412f-a322-3426ba607137" containerName="maas-api"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604315    4577 cpu_manager.go:414] "containerMap: removing container" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604319    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604327    4577 cpu_manager.go:414] "containerMap: removing container" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604335    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604344    4577 cpu_manager.go:414] "containerMap: removing container" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604350    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604364    4577 cpu_manager.go:414] "containerMap: removing container" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerName="payload-processing"
Oct 30 01:57:10.604392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604370    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerName="payload-processing"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604431    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd122b33-6b60-4103-a003-a8d6cdf84f27" containerName="main"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604441    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b62f6b9-8dbb-4fa3-89a1-d76340b4b98a" containerName="maas-api"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604447    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa7f7273-efd5-40f4-830d-a8afe9367c34" containerName="revoke-keys"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604454    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604459    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="f506ab8c-31f5-449c-9662-90d953bcb38c" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604465    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="8083a038-6fca-44e5-bfc8-45af539c67dd" containerName="payload-pre-processing"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604471    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="9efeba0b-699f-4979-b51b-55ca71b66736" containerName="maas-api"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604476    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604482    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a1e68a2-1048-47ae-a00e-3fa2f63fa45d" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604487    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="5087d495-29d9-43a8-872c-10a43a92f05d" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604494    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe335ad9-3734-4a58-a537-f9976621b4c6" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604499    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec96526a-90f6-4096-a8e7-209f0740c279" containerName="payload-pre-processing"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604504    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="61823c04-7687-49d3-95ef-915421584fa0" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604509    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e25ca204-e50f-4b0d-8cb7-b1b34a3fb4fb" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604513    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604519    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="853ec6ff-84b8-4179-b14a-7e50f91534a9" containerName="payload-processing"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604525    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="312cfc2d-795b-4ff1-ba0a-83d5fcbd97e8" containerName="payload-processing"
Oct 30 01:57:10.605507 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.604531    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="515d429f-3299-42c3-96e1-4cbe67ae6cbb" containerName="maas-api"
Oct 30 01:57:10.609286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.609268    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.653110 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.653028    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 01:57:10.653442 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.653426    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:57:10.695992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.695956    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.696206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.696040    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-627t9\" (UniqueName: \"kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.696206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.696085    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.755872 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.755828    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"]
Oct 30 01:57:10.797550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.797516    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.797763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.797612    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.797763 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.797662    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-627t9\" (UniqueName: \"kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.798280 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.798252    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.800246 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.800222    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.840179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.840141    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-627t9\" (UniqueName: \"kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9\") pod \"maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:10.922139 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:10.922057    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:11.228327 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:11.228293    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"]
Oct 30 01:57:11.228462 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:57:11.228423    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65fa0d38_e801_4271_b68d_b85556c81670.slice/crio-d0af95c2ba2b7427005e4e07aa657a853ae2df773f40850567ba776bbea1914d WatchSource:0}: Error finding container d0af95c2ba2b7427005e4e07aa657a853ae2df773f40850567ba776bbea1914d: Status 404 returned error can't find the container with id d0af95c2ba2b7427005e4e07aa657a853ae2df773f40850567ba776bbea1914d
Oct 30 01:57:11.521765 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:11.521735    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg" event={"ID":"65fa0d38-e801-4271-b68d-b85556c81670","Type":"ContainerStarted","Data":"d0af95c2ba2b7427005e4e07aa657a853ae2df773f40850567ba776bbea1914d"}
Oct 30 01:57:11.949098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:11.949008    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:12.530286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:12.530205    4577 generic.go:356] "Generic (PLEG): container finished" podID="65fa0d38-e801-4271-b68d-b85556c81670" containerID="e37025cb4002531efe2b1971195da86f77b7d05c2a591797ce390a14a816819e" exitCode=0
Oct 30 01:57:12.530286 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:12.530250    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg" event={"ID":"65fa0d38-e801-4271-b68d-b85556c81670","Type":"ContainerDied","Data":"e37025cb4002531efe2b1971195da86f77b7d05c2a591797ce390a14a816819e"}
Oct 30 01:57:13.398768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.398729    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:13.516553 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.516521    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:13.682506 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.682482    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:13.727416 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.727341    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:13.831081 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.831051    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca\") pod \"65fa0d38-e801-4271-b68d-b85556c81670\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") "
Oct 30 01:57:13.831306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.831093    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9\" (UniqueName: \"kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9\") pod \"65fa0d38-e801-4271-b68d-b85556c81670\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") "
Oct 30 01:57:13.831306 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.831221    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca\") pod \"65fa0d38-e801-4271-b68d-b85556c81670\" (UID: \"65fa0d38-e801-4271-b68d-b85556c81670\") "
Oct 30 01:57:13.831478 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.831452    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca" pod "65fa0d38-e801-4271-b68d-b85556c81670" (UID: "65fa0d38-e801-4271-b68d-b85556c81670"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:13.833426 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.833400    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca" pod "65fa0d38-e801-4271-b68d-b85556c81670" (UID: "65fa0d38-e801-4271-b68d-b85556c81670"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:57:13.833534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.833434    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9" pod "65fa0d38-e801-4271-b68d-b85556c81670" (UID: "65fa0d38-e801-4271-b68d-b85556c81670"). InnerVolumeSpecName "kube-api-access-627t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:13.932767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.932687    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:13.932767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.932713    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65fa0d38-e801-4271-b68d-b85556c81670-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:13.932767 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:13.932724    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-627t9\" (UniqueName: \"kubernetes.io/projected/65fa0d38-e801-4271-b68d-b85556c81670-kube-api-access-627t9\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:14.545318 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:14.545277    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"
Oct 30 01:57:14.545740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:14.545278    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg" event={"ID":"65fa0d38-e801-4271-b68d-b85556c81670","Type":"ContainerDied","Data":"d0af95c2ba2b7427005e4e07aa657a853ae2df773f40850567ba776bbea1914d"}
Oct 30 01:57:14.545740 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:14.545411    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0af95c2ba2b7427005e4e07aa657a853ae2df773f40850567ba776bbea1914d"
Oct 30 01:57:15.155193 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.155170    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:57:15.245710 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.245680    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.245912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.245754    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.245912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.245816    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.245912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.245881    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.246071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.245968    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.246071 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246048    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.246196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246101    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.246196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246027    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:15.246196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246131    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.246369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246233    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:15.246369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246249    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:15.246369 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246281    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert\") pod \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\" (UID: \"d72b6dbb-08cd-48b1-a0af-167f0cde3191\") "
Oct 30 01:57:15.246900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.246866    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:15.247121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.247061    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.247121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.247085    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.247121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.247102    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.247518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.247248    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:15.248425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.248398    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:15.248425 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.248401    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:57:15.248609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.248582    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "kube-api-access-4mmdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:15.248812 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.248786    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token" pod "d72b6dbb-08cd-48b1-a0af-167f0cde3191" (UID: "d72b6dbb-08cd-48b1-a0af-167f0cde3191"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:15.320069 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.320037    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"]
Oct 30 01:57:15.348333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.348299    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.348333 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.348330    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.348546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.348344    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.348546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.348355    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.348546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.348365    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b6dbb-08cd-48b1-a0af-167f0cde3191-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.348546 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.348377    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mmdf\" (UniqueName: \"kubernetes.io/projected/d72b6dbb-08cd-48b1-a0af-167f0cde3191-kube-api-access-4mmdf\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:15.455269 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.455240    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:15.561776 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.561689    4577 generic.go:356] "Generic (PLEG): container finished" podID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerID="bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca" exitCode=0
Oct 30 01:57:15.562276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.561786    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"
Oct 30 01:57:15.562276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.561789    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" event={"ID":"d72b6dbb-08cd-48b1-a0af-167f0cde3191","Type":"ContainerDied","Data":"bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca"}
Oct 30 01:57:15.562276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.561966    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt" event={"ID":"d72b6dbb-08cd-48b1-a0af-167f0cde3191","Type":"ContainerDied","Data":"381a256126320c22993e0019822241b8dd372b724ec96effaef8ebc04ca7b95a"}
Oct 30 01:57:15.562276 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.562001    4577 scope.go:122] "RemoveContainer" containerID="bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca"
Oct 30 01:57:15.581073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.575749    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:15.589152 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.589131    4577 scope.go:122] "RemoveContainer" containerID="a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"
Oct 30 01:57:15.598056 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.598038    4577 scope.go:122] "RemoveContainer" containerID="bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca"
Oct 30 01:57:15.598361 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:15.598339    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca\": container with ID starting with bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca not found: ID does not exist" containerID="bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca"
Oct 30 01:57:15.598444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.598375    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca"} err="failed to get container status \"bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca\": rpc error: code = NotFound desc = could not find container \"bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca\": container with ID starting with bf8dcb66a9752e15ae22ca2e4447ff7af3db93ba66af578c893ab47e2e899fca not found: ID does not exist"
Oct 30 01:57:15.598444 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.598401    4577 scope.go:122] "RemoveContainer" containerID="a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"
Oct 30 01:57:15.598657 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:15.598641    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a\": container with ID starting with a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a not found: ID does not exist" containerID="a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"
Oct 30 01:57:15.598720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.598665    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a"} err="failed to get container status \"a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a\": rpc error: code = NotFound desc = could not find container \"a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a\": container with ID starting with a9a8b0eb2e41f8809051e021aff3cd33330556587cceec2960ee632dca80cf6a not found: ID does not exist"
Oct 30 01:57:15.624488 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.624452    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"]
Oct 30 01:57:15.628634 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:15.628613    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-bc127f-openshift-default-94fb98bf9-kj9qt"]
Oct 30 01:57:16.109982 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:16.109949    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" path="/var/lib/kubelet/pods/d72b6dbb-08cd-48b1-a0af-167f0cde3191/volumes"
Oct 30 01:57:17.309530 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:17.309499    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:19.102174 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:19.102136    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:20.903438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:20.903407    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:21.047053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:21.047020    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"]
Oct 30 01:57:21.079731 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:21.079701    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"]
Oct 30 01:57:21.188873 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:21.188780    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:21.300213 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:21.300183    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:22.847148 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:22.847109    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:24.395098 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395052    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"]
Oct 30 01:57:24.395518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395466    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy"
Oct 30 01:57:24.395518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395477    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy"
Oct 30 01:57:24.395518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395515    4577 cpu_manager.go:414] "containerMap: removing container" podUID="65fa0d38-e801-4271-b68d-b85556c81670" containerName="revoke-keys"
Oct 30 01:57:24.395518 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395520    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65fa0d38-e801-4271-b68d-b85556c81670" containerName="revoke-keys"
Oct 30 01:57:24.395678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395576    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy"
Oct 30 01:57:24.395678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395585    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="172b3d82-5c75-4c05-8f65-1af8ca1a277d" containerName="istio-proxy"
Oct 30 01:57:24.395678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.395592    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="65fa0d38-e801-4271-b68d-b85556c81670" containerName="revoke-keys"
Oct 30 01:57:24.400294 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.400276    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.417858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.417824    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 01:57:24.419793 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.419769    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"]
Oct 30 01:57:24.430261 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.430239    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmpw6\""
Oct 30 01:57:24.434792 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.434773    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4j5t\" (UniqueName: \"kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.434923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.434800    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.434923 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.434835    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.536121 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.536093    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4j5t\" (UniqueName: \"kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.536290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.536126    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.536290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.536181    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.536964 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.536943    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.538720 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.538692    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.561386 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.561360    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4j5t\" (UniqueName: \"kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t\") pod \"maas-api-revoke-sub-c6945e8d-89spq\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:24.712955 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:24.712876    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:25.016341 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:25.016308    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"]
Oct 30 01:57:25.016651 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:57:25.016627    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ec78f6_5588_45ba_9f31_3577d2bea32f.slice/crio-0f824f57fb74b39f5a6520cb8228cf5031611f7b6a1bcf50871d84d9a0c5df5d WatchSource:0}: Error finding container 0f824f57fb74b39f5a6520cb8228cf5031611f7b6a1bcf50871d84d9a0c5df5d: Status 404 returned error can't find the container with id 0f824f57fb74b39f5a6520cb8228cf5031611f7b6a1bcf50871d84d9a0c5df5d
Oct 30 01:57:25.619655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:25.619547    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq" event={"ID":"96ec78f6-5588-45ba-9f31-3577d2bea32f","Type":"ContainerStarted","Data":"abe407037c14e1d088bc4eb7532e6dae45c81b5e1a5fd64223629b4a2edcc78d"}
Oct 30 01:57:25.619655 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:25.619616    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq" event={"ID":"96ec78f6-5588-45ba-9f31-3577d2bea32f","Type":"ContainerStarted","Data":"0f824f57fb74b39f5a6520cb8228cf5031611f7b6a1bcf50871d84d9a0c5df5d"}
Oct 30 01:57:25.665245 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:25.665195    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq" podStartSLOduration=1.665182205 podStartE2EDuration="1.665182205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:57:25.662722507 +0000 UTC m=+4553.692032738" watchObservedRunningTime="2026-10-30 01:57:25.665182205 +0000 UTC m=+4553.694492435"
Oct 30 01:57:26.047568 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.047511    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf" podUID="414a1c9f-095c-459b-944a-99063ca6f593" containerName="payload-processing" containerID="cri-o://2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28" gracePeriod=25
Oct 30 01:57:26.080308 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.080250    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c" podUID="e2be05f4-9697-411f-a706-f5326cffb0c8" containerName="payload-pre-processing" containerID="cri-o://d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7" gracePeriod=25
Oct 30 01:57:26.338688 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.338658    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 01:57:26.357129 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.357105    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume\") pod \"414a1c9f-095c-459b-944a-99063ca6f593\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") "
Oct 30 01:57:26.357267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.357183    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5\" (UniqueName: \"kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5\") pod \"414a1c9f-095c-459b-944a-99063ca6f593\" (UID: \"414a1c9f-095c-459b-944a-99063ca6f593\") "
Oct 30 01:57:26.357550 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.357523    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume" pod "414a1c9f-095c-459b-944a-99063ca6f593" (UID: "414a1c9f-095c-459b-944a-99063ca6f593"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:26.359551 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.359518    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5" pod "414a1c9f-095c-459b-944a-99063ca6f593" (UID: "414a1c9f-095c-459b-944a-99063ca6f593"). InnerVolumeSpecName "kube-api-access-njhz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:26.360971 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.360950    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 01:57:26.462238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.462156    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume\") pod \"e2be05f4-9697-411f-a706-f5326cffb0c8\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") "
Oct 30 01:57:26.462404 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.462388    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7\" (UniqueName: \"kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7\") pod \"e2be05f4-9697-411f-a706-f5326cffb0c8\" (UID: \"e2be05f4-9697-411f-a706-f5326cffb0c8\") "
Oct 30 01:57:26.462644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.462615    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume" pod "e2be05f4-9697-411f-a706-f5326cffb0c8" (UID: "e2be05f4-9697-411f-a706-f5326cffb0c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:26.462807 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.462785    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2be05f4-9697-411f-a706-f5326cffb0c8-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:26.462912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.462812    4577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/414a1c9f-095c-459b-944a-99063ca6f593-plugins-config-volume\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:26.462912 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.462828    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njhz5\" (UniqueName: \"kubernetes.io/projected/414a1c9f-095c-459b-944a-99063ca6f593-kube-api-access-njhz5\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:26.464738 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.464709    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7" pod "e2be05f4-9697-411f-a706-f5326cffb0c8" (UID: "e2be05f4-9697-411f-a706-f5326cffb0c8"). InnerVolumeSpecName "kube-api-access-jgxk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:26.564173 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.564141    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgxk7\" (UniqueName: \"kubernetes.io/projected/e2be05f4-9697-411f-a706-f5326cffb0c8-kube-api-access-jgxk7\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:26.627633 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.627601    4577 generic.go:356] "Generic (PLEG): container finished" podID="e2be05f4-9697-411f-a706-f5326cffb0c8" containerID="d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7" exitCode=0
Oct 30 01:57:26.628127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.627649    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"
Oct 30 01:57:26.628127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.627698    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c" event={"ID":"e2be05f4-9697-411f-a706-f5326cffb0c8","Type":"ContainerDied","Data":"d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7"}
Oct 30 01:57:26.628127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.627764    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c" event={"ID":"e2be05f4-9697-411f-a706-f5326cffb0c8","Type":"ContainerDied","Data":"6937ebfb52483874a0a1c6ccb7c4fa6b9d9f4d7f61b02b9ed7f8ca057387042f"}
Oct 30 01:57:26.628127 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.627801    4577 scope.go:122] "RemoveContainer" containerID="d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7"
Oct 30 01:57:26.629372 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.629341    4577 generic.go:356] "Generic (PLEG): container finished" podID="96ec78f6-5588-45ba-9f31-3577d2bea32f" containerID="abe407037c14e1d088bc4eb7532e6dae45c81b5e1a5fd64223629b4a2edcc78d" exitCode=0
Oct 30 01:57:26.629487 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.629424    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq" event={"ID":"96ec78f6-5588-45ba-9f31-3577d2bea32f","Type":"ContainerDied","Data":"abe407037c14e1d088bc4eb7532e6dae45c81b5e1a5fd64223629b4a2edcc78d"}
Oct 30 01:57:26.630928 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.630904    4577 generic.go:356] "Generic (PLEG): container finished" podID="414a1c9f-095c-459b-944a-99063ca6f593" containerID="2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28" exitCode=0
Oct 30 01:57:26.631038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.630941    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf" event={"ID":"414a1c9f-095c-459b-944a-99063ca6f593","Type":"ContainerDied","Data":"2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28"}
Oct 30 01:57:26.631038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.630958    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"
Oct 30 01:57:26.631038 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.630984    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf" event={"ID":"414a1c9f-095c-459b-944a-99063ca6f593","Type":"ContainerDied","Data":"048d1b0e747ae9f7cac62b734b2612751c159a6a6d6a747f6e0353bf25967e93"}
Oct 30 01:57:26.638832 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.638805    4577 scope.go:122] "RemoveContainer" containerID="d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7"
Oct 30 01:57:26.639454 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:26.639432    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7\": container with ID starting with d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7 not found: ID does not exist" containerID="d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7"
Oct 30 01:57:26.639524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.639465    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7"} err="failed to get container status \"d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7\": rpc error: code = NotFound desc = could not find container \"d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7\": container with ID starting with d004df8fd4a1369dfcb3ff04ab0ee4cb2df02214e2e0e8f9a6e2947fccda0ca7 not found: ID does not exist"
Oct 30 01:57:26.639524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.639484    4577 scope.go:122] "RemoveContainer" containerID="2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28"
Oct 30 01:57:26.649583 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.649565    4577 scope.go:122] "RemoveContainer" containerID="2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28"
Oct 30 01:57:26.649929 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:26.649904    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28\": container with ID starting with 2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28 not found: ID does not exist" containerID="2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28"
Oct 30 01:57:26.650015 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.649939    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28"} err="failed to get container status \"2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28\": rpc error: code = NotFound desc = could not find container \"2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28\": container with ID starting with 2579420b7b56646e0596b0556cce50615d75377356e884487e932677b25f5b28 not found: ID does not exist"
Oct 30 01:57:26.687264 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.687231    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"]
Oct 30 01:57:26.692678 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.692656    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d5b2b8-756c478bdd-5ddhf"]
Oct 30 01:57:26.773855 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.773805    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"]
Oct 30 01:57:26.780249 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:26.780222    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d5b2b8-59f96bd7cf-lsb4c"]
Oct 30 01:57:27.015805 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:27.015766    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:57:27.681142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.681112    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:27.797897 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.797871    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:27.879014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.878969    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca\") pod \"96ec78f6-5588-45ba-9f31-3577d2bea32f\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") "
Oct 30 01:57:27.879234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.879031    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca\") pod \"96ec78f6-5588-45ba-9f31-3577d2bea32f\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") "
Oct 30 01:57:27.879234 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.879145    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t\" (UniqueName: \"kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t\") pod \"96ec78f6-5588-45ba-9f31-3577d2bea32f\" (UID: \"96ec78f6-5588-45ba-9f31-3577d2bea32f\") "
Oct 30 01:57:27.879915 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.879882    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca" pod "96ec78f6-5588-45ba-9f31-3577d2bea32f" (UID: "96ec78f6-5588-45ba-9f31-3577d2bea32f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:57:27.882053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.882023    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t" pod "96ec78f6-5588-45ba-9f31-3577d2bea32f" (UID: "96ec78f6-5588-45ba-9f31-3577d2bea32f"). InnerVolumeSpecName "kube-api-access-n4j5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:27.882053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.882039    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca" pod "96ec78f6-5588-45ba-9f31-3577d2bea32f" (UID: "96ec78f6-5588-45ba-9f31-3577d2bea32f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:57:27.980810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.980770    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4j5t\" (UniqueName: \"kubernetes.io/projected/96ec78f6-5588-45ba-9f31-3577d2bea32f-kube-api-access-n4j5t\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:27.980810 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.980810    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-openshift-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:27.981073 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:27.980828    4577 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/96ec78f6-5588-45ba-9f31-3577d2bea32f-maas-api-xks-ca\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:28.111037 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:28.111008    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="414a1c9f-095c-459b-944a-99063ca6f593" path="/var/lib/kubelet/pods/414a1c9f-095c-459b-944a-99063ca6f593/volumes"
Oct 30 01:57:28.111438 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:28.111424    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2be05f4-9697-411f-a706-f5326cffb0c8" path="/var/lib/kubelet/pods/e2be05f4-9697-411f-a706-f5326cffb0c8/volumes"
Oct 30 01:57:28.656020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:28.655978    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq" event={"ID":"96ec78f6-5588-45ba-9f31-3577d2bea32f","Type":"ContainerDied","Data":"0f824f57fb74b39f5a6520cb8228cf5031611f7b6a1bcf50871d84d9a0c5df5d"}
Oct 30 01:57:28.656020 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:28.656007    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c6945e8d-89spq"
Oct 30 01:57:28.656248 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:28.656065    4577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f824f57fb74b39f5a6520cb8228cf5031611f7b6a1bcf50871d84d9a0c5df5d"
Oct 30 01:57:30.341543 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.341475    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main" containerID="cri-o://564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239" gracePeriod=45
Oct 30 01:57:30.597799 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.597741    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:57:30.671113 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.671075    4577 generic.go:356] "Generic (PLEG): container finished" podID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerID="564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239" exitCode=0
Oct 30 01:57:30.671314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.671126    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" event={"ID":"7e225ef3-81a7-4fce-a53a-0d1edc73487f","Type":"ContainerDied","Data":"564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239"}
Oct 30 01:57:30.671314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.671133    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"
Oct 30 01:57:30.671314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.671167    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp" event={"ID":"7e225ef3-81a7-4fce-a53a-0d1edc73487f","Type":"ContainerDied","Data":"bd1027d0bf1cc9df1539634b595bd1e1d908c97c9b30c1f8d200f69bf65c8975"}
Oct 30 01:57:30.671314 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.671195    4577 scope.go:122] "RemoveContainer" containerID="564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239"
Oct 30 01:57:30.684114 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.684086    4577 scope.go:122] "RemoveContainer" containerID="564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239"
Oct 30 01:57:30.684537 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:30.684498    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239\": container with ID starting with 564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239 not found: ID does not exist" containerID="564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239"
Oct 30 01:57:30.684630 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.684551    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239"} err="failed to get container status \"564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239\": rpc error: code = NotFound desc = could not find container \"564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239\": container with ID starting with 564f7ff2fd0f024041b0fcf48d5d7c0741fa9c7b2e8bade86facc1a49cc64239 not found: ID does not exist"
Oct 30 01:57:30.709993 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.709961    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache\") pod \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") "
Oct 30 01:57:30.710151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710055    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm\") pod \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") "
Oct 30 01:57:30.710151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710085    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs\") pod \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") "
Oct 30 01:57:30.710151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710120    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79\" (UniqueName: \"kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79\") pod \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") "
Oct 30 01:57:30.710344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710162    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir\") pod \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") "
Oct 30 01:57:30.710344 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710251    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache" pod "7e225ef3-81a7-4fce-a53a-0d1edc73487f" (UID: "7e225ef3-81a7-4fce-a53a-0d1edc73487f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:30.710576 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710552    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir" pod "7e225ef3-81a7-4fce-a53a-0d1edc73487f" (UID: "7e225ef3-81a7-4fce-a53a-0d1edc73487f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:30.710658 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710571    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home\") pod \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\" (UID: \"7e225ef3-81a7-4fce-a53a-0d1edc73487f\") "
Oct 30 01:57:30.710761 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.710742    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home" pod "7e225ef3-81a7-4fce-a53a-0d1edc73487f" (UID: "7e225ef3-81a7-4fce-a53a-0d1edc73487f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:30.711179 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.711157    4577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-model-cache\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:30.711299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.711185    4577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tmp-dir\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:30.711299 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.711200    4577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-home\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:30.712753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.712716    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs" pod "7e225ef3-81a7-4fce-a53a-0d1edc73487f" (UID: "7e225ef3-81a7-4fce-a53a-0d1edc73487f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:57:30.713236 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.713213    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm" pod "7e225ef3-81a7-4fce-a53a-0d1edc73487f" (UID: "7e225ef3-81a7-4fce-a53a-0d1edc73487f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:57:30.713365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.713345    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79" pod "7e225ef3-81a7-4fce-a53a-0d1edc73487f" (UID: "7e225ef3-81a7-4fce-a53a-0d1edc73487f"). InnerVolumeSpecName "kube-api-access-xnd79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:57:30.812267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.812233    4577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e225ef3-81a7-4fce-a53a-0d1edc73487f-dshm\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:30.812267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.812263    4577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e225ef3-81a7-4fce-a53a-0d1edc73487f-tls-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:30.812267 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:30.812274    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnd79\" (UniqueName: \"kubernetes.io/projected/7e225ef3-81a7-4fce-a53a-0d1edc73487f-kube-api-access-xnd79\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:57:31.020540 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:31.020499    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"]
Oct 30 01:57:31.026593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:31.026565    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-d5b2b8/distinct-2-w4-d5b2b8-kserve-585f64c995-qpcwp"]
Oct 30 01:57:32.110029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:32.109994    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" path="/var/lib/kubelet/pods/7e225ef3-81a7-4fce-a53a-0d1edc73487f/volumes"
Oct 30 01:57:39.683013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:39.682887    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:40.090238 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:40.090202    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:40.206132 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:40.206069    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:40.314837 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:40.314803    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"]
Oct 30 01:57:40.327640 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:40.327609    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-d5b2b8-82pjg"]
Oct 30 01:57:40.807312 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:40.807275    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:42.120206 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:42.120165    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65fa0d38-e801-4271-b68d-b85556c81670" path="/var/lib/kubelet/pods/65fa0d38-e801-4271-b68d-b85556c81670/volumes"
Oct 30 01:57:46.939650 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:46.939614    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:47.090728 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:47.090671    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:47.312168 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:47.312130    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:48.324900 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:48.324867    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-976b87cf6-rjtkz_c86df6e2-f26b-4af0-976a-c4f26169dcb8/authorino/0.log"
Oct 30 01:57:48.586472 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:48.586398    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-66ztg_8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3/manager/0.log"
Oct 30 01:57:48.785609 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:48.785581    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-nrc8l_e49ab448-f19b-475e-add1-ad820ccc0eaa/manager/0.log"
Oct 30 01:57:49.359142 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:49.359111    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-pwsx2_1bd6ce02-54e6-45af-9875-a50830402b33/manager/0.log"
Oct 30 01:57:49.752217 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:49.752185    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-msqjs_338395d0-2720-4943-8adb-cd3a3474526b/manager/0.log"
Oct 30 01:57:52.728216 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:52.728172    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:53.029254 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:53.029162    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:53.068937 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:53.068900    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-976b87cf6-rjtkz_c86df6e2-f26b-4af0-976a-c4f26169dcb8/authorino/0.log"
Oct 30 01:57:53.142813 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:53.142778    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:53.352794 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:53.352703    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:53.601429 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:53.601386    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:55.036786 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:55.036755    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:57:57.012577 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:57:57.012537    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:57:59.474502 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:57:59.474464    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:07.024719 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:07.024687    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-a8e3c6c5-s9jqt_aa7f7273-efd5-40f4-830d-a8afe9367c34/revoke-keys/0.log"
Oct 30 01:58:07.394141 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:07.394065    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c6945e8d-89spq_96ec78f6-5588-45ba-9f31-3577d2bea32f/revoke-keys/0.log"
Oct 30 01:58:07.949046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:07.949019    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-nqbth_5cef7ef0-fef6-47df-8a73-531458d5a43e/postgres/0.log"
Oct 30 01:58:08.591501 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:08.591469    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:08.721741 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:08.721711    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:09.363769 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:09.363737    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-976b87cf6-rjtkz_c86df6e2-f26b-4af0-976a-c4f26169dcb8/authorino/0.log"
Oct 30 01:58:09.584558 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:09.584525    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-66ztg_8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3/manager/0.log"
Oct 30 01:58:09.781458 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:09.781433    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-nrc8l_e49ab448-f19b-475e-add1-ad820ccc0eaa/manager/0.log"
Oct 30 01:58:10.248096 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:10.248064    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-pwsx2_1bd6ce02-54e6-45af-9875-a50830402b33/manager/0.log"
Oct 30 01:58:10.635649 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:10.635566    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-msqjs_338395d0-2720-4943-8adb-cd3a3474526b/manager/0.log"
Oct 30 01:58:11.371083 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:11.371046    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7c689579c-js4rj_ff1714b0-728c-4d2c-8577-c657d2de0c77/istio-proxy/0.log"
Oct 30 01:58:11.549013 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:11.548982    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-f8t97_93475d5e-0573-4331-93a9-15205495cb83/discovery/0.log"
Oct 30 01:58:12.081371 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:12.081340    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-fcb75c45f-4x4jk_d07ac1e5-f1b2-4018-9cfb-1f71f672c381/istio-proxy/0.log"
Oct 30 01:58:12.253579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:12.253547    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-fcb75c45f-69fm9_6c0d7936-d648-4a3d-bf12-3ae6d46503e2/istio-proxy/0.log"
Oct 30 01:58:12.799078 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:12.799049    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-fcb75c45f-x8q6j_8a4425a4-df44-4ba0-adce-5e733e2e6285/istio-proxy/0.log"
Oct 30 01:58:12.986672 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:12.986638    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p_ec76aee0-2b4c-4ad0-b742-5bf48b05deea/istio-proxy/0.log"
Oct 30 01:58:14.836134 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:14.836108    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-nw82l_f0f0cce1-38e9-4093-a90f-bee3ad2b674f/main/0.log"
Oct 30 01:58:15.028878 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:15.028828    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cvz4b_e8c30c7a-2aeb-4913-b607-f752f1bb1911/main/0.log"
Oct 30 01:58:15.205681 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:15.205602    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-wc2r5_b48c6744-79c5-46c4-938d-f7f943ea9a16/main/0.log"
Oct 30 01:58:17.550749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:17.550713    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:17.661595 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:17.661561    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:25.543882 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:25.543820    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:25.673005 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:25.672969    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:27.013020 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:58:27.012977    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:58:27.831185 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:27.831150    4577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log" limit=1024
Oct 30 01:58:30.830960 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:30.830927    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7pfth_f7088947-2eec-480b-9df3-285c4e6b2b59/global-pull-secret-syncer/0.log"
Oct 30 01:58:31.797392 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:31.797360    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2ttqh_c9c1e69b-d783-4731-8eba-61f21232cb31/konnectivity-agent/0.log"
Oct 30 01:58:32.480180 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:32.480144    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-17.ec2.internal_ab07c2e2299f7ce9c9ccf7ec1f988b83/haproxy/0.log"
Oct 30 01:58:39.982058 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:39.982021    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"]
Oct 30 01:58:39.982490 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:39.982253    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerName="istio-proxy" containerID="cri-o://8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e" gracePeriod=30
Oct 30 01:58:39.986290 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:39.986265    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"]
Oct 30 01:58:39.986482 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:39.986457    4577 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerName="istio-proxy" containerID="cri-o://b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261" gracePeriod=30
Oct 30 01:58:40.250593 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:40.250517    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-976b87cf6-rjtkz_c86df6e2-f26b-4af0-976a-c4f26169dcb8/authorino/0.log"
Oct 30 01:58:40.356018 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:40.355993    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-66ztg_8377b8b6-6ea1-4d49-9bf2-c57cc68e27d3/manager/0.log"
Oct 30 01:58:40.477591 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:40.477561    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-nrc8l_e49ab448-f19b-475e-add1-ad820ccc0eaa/manager/0.log"
Oct 30 01:58:40.808498 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:40.808465    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-pwsx2_1bd6ce02-54e6-45af-9875-a50830402b33/manager/0.log"
Oct 30 01:58:41.495082 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:41.494989    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-msqjs_338395d0-2720-4943-8adb-cd3a3474526b/manager/0.log"
Oct 30 01:58:43.119046 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.118973    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/alertmanager/0.log"
Oct 30 01:58:43.239485 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.239427    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/config-reloader/0.log"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.277362    4577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"]
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278380    4577 cpu_manager.go:414] "containerMap: removing container" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278402    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278437    4577 cpu_manager.go:414] "containerMap: removing container" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278446    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278474    4577 cpu_manager.go:414] "containerMap: removing container" podUID="414a1c9f-095c-459b-944a-99063ca6f593" containerName="payload-processing"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278481    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="414a1c9f-095c-459b-944a-99063ca6f593" containerName="payload-processing"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278502    4577 cpu_manager.go:414] "containerMap: removing container" podUID="96ec78f6-5588-45ba-9f31-3577d2bea32f" containerName="revoke-keys"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278509    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96ec78f6-5588-45ba-9f31-3577d2bea32f" containerName="revoke-keys"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278535    4577 cpu_manager.go:414] "containerMap: removing container" podUID="e2be05f4-9697-411f-a706-f5326cffb0c8" containerName="payload-pre-processing"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278542    4577 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2be05f4-9697-411f-a706-f5326cffb0c8" containerName="payload-pre-processing"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278711    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e225ef3-81a7-4fce-a53a-0d1edc73487f" containerName="main"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278737    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="414a1c9f-095c-459b-944a-99063ca6f593" containerName="payload-processing"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278752    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="96ec78f6-5588-45ba-9f31-3577d2bea32f" containerName="revoke-keys"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278762    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2be05f4-9697-411f-a706-f5326cffb0c8" containerName="payload-pre-processing"
Oct 30 01:58:43.281774 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.278780    4577 memory_manager.go:365] "RemoveStaleState removing state" podUID="d72b6dbb-08cd-48b1-a0af-167f0cde3191" containerName="istio-proxy"
Oct 30 01:58:43.284598 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.282996    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.299611 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.299581    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"]
Oct 30 01:58:43.335119 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.335081    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srxqq\" (UniqueName: \"kubernetes.io/projected/95c24d72-e4f9-4bf1-af9a-d329958f9b72-kube-api-access-srxqq\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.335297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.335128    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-sys\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.335297 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.335245    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-proc\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.335390 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.335302    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-podres\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.335439 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.335387    4577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-lib-modules\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.339779 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.339763    4577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qqxng\"/\"default-dockercfg-jkqcz\""
Oct 30 01:58:43.370505 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.370437    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/kube-rbac-proxy-web/0.log"
Oct 30 01:58:43.396753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.396724    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqxng\"/\"kube-root-ca.crt\""
Oct 30 01:58:43.396753 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.396734    4577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqxng\"/\"openshift-service-ca.crt\""
Oct 30 01:58:43.436839 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.436806    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srxqq\" (UniqueName: \"kubernetes.io/projected/95c24d72-e4f9-4bf1-af9a-d329958f9b72-kube-api-access-srxqq\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.436892    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-sys\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.436948    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-proc\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437014 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.437006    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-podres\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.437014    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-proc\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.437043    4577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-lib-modules\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.437089    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-podres\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.437014    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-sys\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.437144 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.437092    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95c24d72-e4f9-4bf1-af9a-d329958f9b72-lib-modules\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.462638 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.462605    4577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srxqq\" (UniqueName: \"kubernetes.io/projected/95c24d72-e4f9-4bf1-af9a-d329958f9b72-kube-api-access-srxqq\") pod \"perf-node-gather-daemonset-bvdcb\" (UID: \"95c24d72-e4f9-4bf1-af9a-d329958f9b72\") " pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.494050 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.494024    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/kube-rbac-proxy/0.log"
Oct 30 01:58:43.602251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.602221    4577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:43.612188 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.612163    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/kube-rbac-proxy-metric/0.log"
Oct 30 01:58:43.740064 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.740031    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/prom-label-proxy/0.log"
Oct 30 01:58:43.863240 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.863214    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7f8ea78-d9c6-4774-8938-0e273c1e2ad1/init-config-reloader/0.log"
Oct 30 01:58:43.903295 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:43.903268    4577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"]
Oct 30 01:58:43.905330 ip-10-0-141-17 kubenswrapper[4577]: W1030 01:58:43.905303    4577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod95c24d72_e4f9_4bf1_af9a_d329958f9b72.slice/crio-d88c60c016c3de3de33ac569f31f3ecf8cef9abc980cf021c80e052b2db61afc WatchSource:0}: Error finding container d88c60c016c3de3de33ac569f31f3ecf8cef9abc980cf021c80e052b2db61afc: Status 404 returned error can't find the container with id d88c60c016c3de3de33ac569f31f3ecf8cef9abc980cf021c80e052b2db61afc
Oct 30 01:58:44.078451 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.078418    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb" event={"ID":"95c24d72-e4f9-4bf1-af9a-d329958f9b72","Type":"ContainerStarted","Data":"4f59377c2634d651141e36e238e161d616d25f28980f17a749558b4cc00977c4"}
Oct 30 01:58:44.078625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.078479    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb" event={"ID":"95c24d72-e4f9-4bf1-af9a-d329958f9b72","Type":"ContainerStarted","Data":"d88c60c016c3de3de33ac569f31f3ecf8cef9abc980cf021c80e052b2db61afc"}
Oct 30 01:58:44.078625 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.078512    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:44.122237 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.122187    4577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb" podStartSLOduration=1.122173761 podStartE2EDuration="1.122173761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 01:58:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:58:44.12174862 +0000 UTC m=+4632.151058854" watchObservedRunningTime="2026-10-30 01:58:44.122173761 +0000 UTC m=+4632.151483992"
Oct 30 01:58:44.151579 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.151549    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-prwfk_99689af2-475f-4378-80b2-f9a655ca86fd/kube-state-metrics/0.log"
Oct 30 01:58:44.276251 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.276213    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-prwfk_99689af2-475f-4378-80b2-f9a655ca86fd/kube-rbac-proxy-main/0.log"
Oct 30 01:58:44.401524 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:44.401491    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-prwfk_99689af2-475f-4378-80b2-f9a655ca86fd/kube-rbac-proxy-self/0.log"
Oct 30 01:58:45.172409 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.172384    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l5s6p_0d9efebc-6819-40df-a72e-74a4c9766174/node-exporter/0.log"
Oct 30 01:58:45.250791 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.250766    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:58:45.269803 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.269740    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:58:45.295196 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.295169    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l5s6p_0d9efebc-6819-40df-a72e-74a4c9766174/kube-rbac-proxy/0.log"
Oct 30 01:58:45.356771 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356736    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356790    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356816    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356833    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356882    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356913    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356931    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356952    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.356979 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.356971    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357011    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357041    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357058    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357098    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357128    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357164    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357193    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.357338 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357260    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy\") pod \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\" (UID: \"6c0d7936-d648-4a3d-bf12-3ae6d46503e2\") "
Oct 30 01:58:45.357992 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.357350    4577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy\") pod \"8a4425a4-df44-4ba0-adce-5e733e2e6285\" (UID: \"8a4425a4-df44-4ba0-adce-5e733e2e6285\") "
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.358516    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.358895    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.358917    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359150    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359389    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359397    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359618    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359694    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359773    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.359906    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.360001    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.360151 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.360001    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:58:45.361156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.360353    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.361156 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.360911    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:58:45.361379 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.361358    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:58:45.361644 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.361628    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "kube-api-access-hg8qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:58:45.362365 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.362342    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy" pod "6c0d7936-d648-4a3d-bf12-3ae6d46503e2" (UID: "6c0d7936-d648-4a3d-bf12-3ae6d46503e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:58:45.363133 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.363109    4577 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc" pod "8a4425a4-df44-4ba0-adce-5e733e2e6285" (UID: "8a4425a4-df44-4ba0-adce-5e733e2e6285"). InnerVolumeSpecName "kube-api-access-svbhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:58:45.418279 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.418247    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l5s6p_0d9efebc-6819-40df-a72e-74a4c9766174/init-textfile/0.log"
Oct 30 01:58:45.458911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458830    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.458911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458898    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.458911 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458913    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458925    4577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-token\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458938    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458950    4577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-envoy\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458960    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458973    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458985    4577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a4425a4-df44-4ba0-adce-5e733e2e6285-istiod-ca-cert\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.458997    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459010    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg8qc\" (UniqueName: \"kubernetes.io/projected/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-kube-api-access-hg8qc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459025    4577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-data\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459039    4577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a4425a4-df44-4ba0-adce-5e733e2e6285-istio-podinfo\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459051    4577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-workload-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459064    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459077    4577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svbhc\" (UniqueName: \"kubernetes.io/projected/8a4425a4-df44-4ba0-adce-5e733e2e6285-kube-api-access-svbhc\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459090    4577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a4425a4-df44-4ba0-adce-5e733e2e6285-credential-socket\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:45.459357 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:45.459104    4577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c0d7936-d648-4a3d-bf12-3ae6d46503e2-workload-certs\") on node \"ip-10-0-141-17.ec2.internal\" DevicePath \"\""
Oct 30 01:58:46.090399 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.090365    4577 generic.go:356] "Generic (PLEG): container finished" podID="8a4425a4-df44-4ba0-adce-5e733e2e6285" containerID="8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e" exitCode=0
Oct 30 01:58:46.090592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.090415    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"
Oct 30 01:58:46.090592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.090451    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" event={"ID":"8a4425a4-df44-4ba0-adce-5e733e2e6285","Type":"ContainerDied","Data":"8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e"}
Oct 30 01:58:46.090592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.090508    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j" event={"ID":"8a4425a4-df44-4ba0-adce-5e733e2e6285","Type":"ContainerDied","Data":"7b87b0cdf04bd321064602ac8af17b3fba2a535428d1e6dc66a4e7a29772df92"}
Oct 30 01:58:46.090592 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.090541    4577 scope.go:122] "RemoveContainer" containerID="8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e"
Oct 30 01:58:46.091858 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.091822    4577 generic.go:356] "Generic (PLEG): container finished" podID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" containerID="b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261" exitCode=0
Oct 30 01:58:46.091966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.091886    4577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"
Oct 30 01:58:46.091966 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.091886    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" event={"ID":"6c0d7936-d648-4a3d-bf12-3ae6d46503e2","Type":"ContainerDied","Data":"b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261"}
Oct 30 01:58:46.092086 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.092002    4577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9" event={"ID":"6c0d7936-d648-4a3d-bf12-3ae6d46503e2","Type":"ContainerDied","Data":"9bd0ca7e094245e49eeb124bc744dd7c3e37ac675fd3281d99a1512865fdce59"}
Oct 30 01:58:46.101730 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.101669    4577 scope.go:122] "RemoveContainer" containerID="8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e"
Oct 30 01:58:46.102064 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:58:46.102036    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e\": container with ID starting with 8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e not found: ID does not exist" containerID="8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e"
Oct 30 01:58:46.102182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.102074    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e"} err="failed to get container status \"8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e\": rpc error: code = NotFound desc = could not find container \"8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e\": container with ID starting with 8d8723c392b6864786ae17b8254efaa51cfda2251ff8bd971e749dc21d4ba96e not found: ID does not exist"
Oct 30 01:58:46.102182 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.102097    4577 scope.go:122] "RemoveContainer" containerID="b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261"
Oct 30 01:58:46.112768 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.112566    4577 scope.go:122] "RemoveContainer" containerID="b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261"
Oct 30 01:58:46.113053 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:58:46.112879    4577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261\": container with ID starting with b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261 not found: ID does not exist" containerID="b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261"
Oct 30 01:58:46.113053 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.112908    4577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261"} err="failed to get container status \"b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261\": rpc error: code = NotFound desc = could not find container \"b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261\": container with ID starting with b675d84c6f43afb5e7496c122ab1d7f602f1710f2b24f95283c6475327bc0261 not found: ID does not exist"
Oct 30 01:58:46.168831 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.168797    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"]
Oct 30 01:58:46.182949 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.182919    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x8q6j"]
Oct 30 01:58:46.275749 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.275665    4577 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"]
Oct 30 01:58:46.289534 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:46.289502    4577 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-69fm9"]
Oct 30 01:58:47.847084 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:47.847057    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9f98946d-bhq9t_93128ccb-ef11-4510-8f32-9e7e0c33a6fd/telemeter-client/0.log"
Oct 30 01:58:47.967228 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:47.967203    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9f98946d-bhq9t_93128ccb-ef11-4510-8f32-9e7e0c33a6fd/reload/0.log"
Oct 30 01:58:48.093029 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:48.093002    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9f98946d-bhq9t_93128ccb-ef11-4510-8f32-9e7e0c33a6fd/kube-rbac-proxy/0.log"
Oct 30 01:58:48.110017 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:48.109944    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c0d7936-d648-4a3d-bf12-3ae6d46503e2" path="/var/lib/kubelet/pods/6c0d7936-d648-4a3d-bf12-3ae6d46503e2/volumes"
Oct 30 01:58:48.110399 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:48.110386    4577 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a4425a4-df44-4ba0-adce-5e733e2e6285" path="/var/lib/kubelet/pods/8a4425a4-df44-4ba0-adce-5e733e2e6285/volumes"
Oct 30 01:58:50.095070 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:50.095044    4577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qqxng/perf-node-gather-daemonset-bvdcb"
Oct 30 01:58:53.143111 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:53.143079    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b76f84fd4-lqkj8_7f0f01d3-84a1-4200-993f-ab77f7b90113/console/0.log"
Oct 30 01:58:57.013492 ip-10-0-141-17 kubenswrapper[4577]: E1030 01:58:57.013450    4577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-2v4fv" containerName="extractor"
Oct 30 01:58:58.115446 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:58.115420    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hmkqx_ac84b9b8-dee8-4ecf-acd0-6b093753b60c/dns/0.log"
Oct 30 01:58:58.488995 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:58.488971    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hmkqx_ac84b9b8-dee8-4ecf-acd0-6b093753b60c/kube-rbac-proxy/0.log"
Oct 30 01:58:59.178219 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:59.178189    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pc8xn_f29d1b61-cd81-4277-af77-a44af9cc5237/dns-node-resolver/0.log"
Oct 30 01:58:59.260954 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:58:59.260924    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7c689579c-js4rj_ff1714b0-728c-4d2c-8577-c657d2de0c77/istio-proxy/0.log"
Oct 30 01:59:00.720674 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:00.720638    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-577c49dcc4-dvt9d_07421265-50a3-4221-9ba0-13605a6fe3f3/registry/0.log"
Oct 30 01:59:00.822195 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:00.822162    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5q95w_b995c294-397e-41e0-8a66-6e9b225ffd34/node-ca/0.log"
Oct 30 01:59:01.294917 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:01.294880    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7c689579c-js4rj_ff1714b0-728c-4d2c-8577-c657d2de0c77/istio-proxy/0.log"
Oct 30 01:59:02.081004 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:02.080962    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-f8t97_93475d5e-0573-4331-93a9-15205495cb83/discovery/0.log"
Oct 30 01:59:02.264572 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:02.264542    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-fcb75c45f-4x4jk_d07ac1e5-f1b2-4018-9cfb-1f71f672c381/istio-proxy/0.log"
Oct 30 01:59:02.479104 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:02.479071    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-74987c9b47-w6g5p_ec76aee0-2b4c-4ad0-b742-5bf48b05deea/istio-proxy/0.log"
Oct 30 01:59:02.484661 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:02.484640    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7c689579c-js4rj_ff1714b0-728c-4d2c-8577-c657d2de0c77/istio-proxy/0.log"
Oct 30 01:59:03.478997 ip-10-0-141-17 kubenswrapper[4577]: I1030 01:59:03.478950    4577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7c689579c-js4rj_ff1714b0-728c-4d2c-8577-c657d2de0c77/istio-proxy/0.log"